Revisiting local large language models — what’s new, what actually runs on 48 GB, and whether you can trust what they say.
A year ago, “run a serious LLM on your own machine” meant a noisy tower with a £1,500 graphics card and a lot of patience. In late 2026 it means a Mac mini that sits silently on your desk. We put one to the test — an Apple M4 Pro with 48 GB of unified memory — downloaded the current crop of open-weight models, and measured what they actually do. No cloud. No API key. No per-token bill. Everything below ran fully offline on a machine you could carry home in one hand.
Why bother running models locally?
Four reasons, and they all still hold:
- Privacy — your prompts and data never leave the device. For client work, that’s not a nice-to-have.
- Cost — you pay once for the hardware, then inference is free. No metered API.
- Offline — it works on a train, on a plane, in a locked-down environment.
- Control — no silent model swaps, no rate limits, no deprecations.
The trade-off used to be capability. That gap has narrowed dramatically — which is the whole point of revisiting this now.
The one number that matters: unified memory
Here’s the thing most people get wrong about Apple Silicon for AI. On a normal PC, the model has to fit in your graphics card’s VRAM — usually 8–24 GB, and expensive to grow. On a Mac, the CPU and GPU share one pool of memory. So a 48 GB Mac mini effectively gives the GPU far more room than a consumer graphics card ever would.
The rule of thumb: at 4-bit quantization (the standard for local use), a model needs roughly 0.6 GB of memory per billion parameters. So:
| Model size | Memory needed | Fits in 48 GB? |
|---|---|---|
| 14B | ~9 GB | Easily |
| 24–27B | ~14–17 GB | Comfortably |
| 32B | ~20 GB | Comfortably — the sweet spot |
| 70B | ~43 GB | Marginal — it loads, but it’s tight |
| 109B+ (MoE) | ~60 GB+ | No |
Storage is the easy part: each model is downloaded once at roughly its in-memory size, and our full test set was about 100 GB — nothing on a modern SSD. Keep them on the internal drive, though: the whole model is read into memory on first use, so a slow external disk makes every “cold start” crawl.
The setup: two commands
brew install ollama
brew services start ollamaThat’s it. Then ollama pull <model> downloads one, and ollama run <model> chats with it. Ollama handles the rest.
The models we tested
The local scene moved on in 2026. The names that mattered a year ago have been replaced:
- Qwen3 — the new safe default. Great at chat, code, and tool use; the natural “brain” for an agent.
- Gemma 3 (Google) — the family best-tuned for Apple Silicon.
- DeepSeek-R1 — a dedicated reasoning model that “thinks” before answering.
- Mistral Small — fast and long-context.
- Phi-4 (Microsoft) — tiny but punches far above its weight.
- Llama 3.3 70B — included deliberately to find the ceiling of a 48 GB machine.
The results
We gave every model the same prompt and measured generation speed on the M4 Pro. For context, comfortable reading speed is around 5 tokens/sec — so anything above that feels responsive.
| Model | Size | tokens/sec | Feel |
|---|---|---|---|
| Phi-4 14B | 9 GB | 24.9 | Instant |
| Mistral Small 24B | 14 GB | 16.8 | Very fast |
| Gemma 3 27B | 17 GB | 13.7 | Fast |
| DeepSeek-R1 32B | 19 GB | 11.8 | Comfortable, “thinks” first |
| Qwen3 32B | 20 GB | 11.6 | Comfortable |
The pattern is exactly what the hardware predicts: smaller models run faster, and even the 32B “sweet spot” models generate text faster than you can read it. DeepSeek-R1 is the interesting one — it’s a reasoning model that works through a problem step-by-step before answering, so it produces far more tokens per request, but still at a comfortable pace.
Where 48 GB hits its wall
We deliberately tried to push past the sweet spot with Llama 3.3 70B. This is where the honesty comes in: a 70B model at 4-bit wants about 43 GB of memory. On a 48 GB machine, once you account for macOS and room for the conversation itself, you’re right at the edge — the model effectively wants the whole machine. In practice it tips into swapping to disk, where generation speed falls off a cliff (independent testing puts Llama 3.3 70B at roughly 8–12 tokens/sec on a 64 GB mini — and 48 GB has less headroom still).
The takeaway is clear: the sweet spot for a 48 GB Mac mini is 27–32B models. They’re fast, genuinely capable, and leave headroom. If a 70B-class model is your daily driver, buy 64 GB or more — and note that the newest 100B+ “mixture-of-experts” flagships (like Llama 4 Scout) don’t fit 48 GB at all, because they still need memory for every expert even though only a fraction runs at once.
But can you trust what it says?
Speed is the easy question. Here’s the harder one.
We gave all five models the same practical task — estimate the annual carbon a small woodland captures (150 oak trees plus two hectares of new native woodland) — and asked for a single number. The answers:
| Model | Its answer |
|---|---|
| Gemma 3 27B | 7.7 tonnes CO₂/yr |
| Qwen3 32B | 8.5 tonnes CO₂/yr |
| Phi-4 14B | 12.6 tonnes CO₂/yr |
| Mistral Small 24B | 17.0 tonnes CO₂/yr |
| DeepSeek-R1 32B | 27.3 tonnes CO₂/yr |
Same question, same five models — 7.7 to 27.3 tonnes, a 3.5× spread. Some couldn’t even keep their own arithmetic internally consistent.
This isn’t a knock on local models specifically — cloud models do it too. It’s the fundamental thing to understand about any LLM: it’s a brilliant reasoning engine, but a shaky source of facts. Ask it to recall a specific sequestration rate and it will confidently hand you a plausible-sounding number that’s often wrong.
The fix is not to train your own model — that’s expensive and usually the wrong tool. It’s RAG (retrieval-augmented generation): give the model the authoritative numbers at the moment it answers — your Woodland Carbon Code tables, your methodology, your species rates — and let it do only the arithmetic against fixed facts. Give it the facts; let it do the thinking. That single change collapses the 3.5× spread into one number you can actually trust.
That’s the real lesson from running these models locally: the model is the engine, your knowledge base is the fuel. Get that split right and a private, offline Mac mini becomes a genuinely useful tool — not just a fast one.
So — are local models good enough now?
Honestly? For a lot of real work, yes. The 27–32B models handle coding, summarizing, drafting, and agent-style tasks well. The remaining gap to the big cloud models (Claude, GPT) shows up on the hardest reasoning and very-long-context jobs — but for privacy, offline use, and zero marginal cost, a local model on a Mac mini is a genuinely practical tool, not a toy.
The quiet little box on the desk turns out to be a capable, private AI server. A year ago that sentence would have needed an asterisk. It doesn’t anymore.
Tested on a Mac mini (Apple M4 Pro, 48 GB unified memory, macOS 26) using Ollama. All models run fully offline.