Choosing and sizing your model

Here’s the truth nobody likes to hear: you don’t choose your model, your machine does. You can dream of running the biggest open-weight monster on the planet, but if you have 32 GB of RAM, it’ll never launch. The good news is that in 2026 the local landscape has become excellent, as long as you aim right. We’ll first lay down the rule that decides everything (sizing), then look at the catalog, and finally settle the real question: when your mini-PC is enough, and when Claude stays in front.

The sizing rule (the one that decides everything)

Before model names, physics. A model is billions of parameters (the “weights”), and each one takes up space in memory. The mental formula fits on one line:

RAM needed ≈ (number of parameters × bytes per weight) + the context cache (the KV cache)

The bytes per weight depend on precision (we’ll come back to that with quantization). In practice, in Q4_K_M, the standard format, count ~0.6 GB per billion parameters (the rule Quelle IA follows, explained in its “Comment on compte” section, in French). Add 0.5 to 2 GB for the system, and +30 to 50% if you work with a long context or several parallel requests.

A few reference points in Q4_K_M, to train your eye: a Llama 8B fits in ~6 GB, a dense 32B model in ~19 GB, a Llama 70B demands 40-43 GB. And here’s the table to keep handy, by memory tier. These are Quelle IA’s picks in the 28 September 2026 edition (in French); they change often, so check your machine’s page before downloading 20 GB.

For your exact setup: each machine’s page (for example the 128 GB Ryzen AI Max+ 395 or the 32 GB Mac mini M6), or Find my model in four questions (in French).

# what the integrated GPU took from the system, in bytes
cat /sys/class/drm/card*/device/mem_info_gtt_used

# reserve 28 GB of RAM for the rest of the machine (value in bytes)
# set this in the Ollama service environment
OLLAMA_GPU_OVERHEAD=30064771072

Without that setting, Ollama sizes its layers as if it were alone on the machine. It sees a 128 GB GPU, so it accepts a 65 GB model on a 94 GB machine that hosts thirty other services.

The shortcut: let a tool size it for you

The whole rule we just laid down (counting GBs, gauging the quant, spotting MoEs), a small open-source tool does it for you in two seconds: llmfit (MIT license, repo AlexsJones/llmfit). It scans your machine (RAM, CPU cores, VRAM, multi-GPU) and the installed runtimes (Ollama, llama.cpp, LM Studio, MLX, Docker Model Runner), cross-references that with a catalog of models and quants, and gives you a scored ranking: what fits, what will run fast, and the quant to aim for. Enough to avoid pulling 20 GB of model for nothing.

Both tools tell you what fits. To know what is best rated among what fits, task by task (code, agents, French…), use Quelle IA, machine by machine (in French): 64 real machines, with dated prices.

The 2026 model landscape

The headline: Chinese labs dominate open-weight (Alibaba, DeepSeek, Z.ai, Moonshot, MiniMax, Xiaomi), with Google (Gemma) and OpenAI (gpt-oss) as the main American options. Two silhouettes stand out. The giants (from a few hundred billion up to 2.8 T total parameters) for servers or clusters of machines. And the compact models (around 25-30B, dense or MoE) which actually run on a mini-PC. It’s this second family that interests us.

Here are the safe picks as of 28 September 2026, according to Quelle IA (in French):

The first choice on a mini-PC is qwen3.8:27b, as soon as it fits. If speed matters more than the last few quality points, gemma4:26b answers much faster thanks to its 4B active parameters. And on a 128 GB machine running agents, gpt-oss:120b is worth a try. The ranking moves every week: local models ranking, all open models (including those that won’t fit at home), and Compare to put two models side by side (all in French).

Quantization, plainly

You’ve seen names like Q4_K_M go by. Let’s decode them. GGUF is the file format for local models, and the suffix indicates the precision: how many bits per weight you keep. Fewer bits = lighter file = it fits in less RAM. The trade-off is a small loss of quality.

Q4_K_M weighs about 30% of the original FP16 model for only ~2-3% loss. For code specifically, it holds up very well, moving to Q5 or Q6 doesn’t perceptibly improve code generation. You only drop below Q4 when forced, and you feel it fast (code and reasoning are the first to suffer).

Wiring the right quant into Ollama

In practice, it’s simple: with Ollama, the tag already encodes the quant. You choose your precision by writing it into the name.

Honestly: local or Claude?

We won’t sell you a dream. Here’s where local stands in Quelle IA’s 28 September 2026 edition, unfiltered.

For bounded and private tasks, short-window generation, refactoring, autocomplete, targeted debugging, local is genuinely competitive. It’s private, it’s free to use, and the benchmark gap has narrowed sharply (GLM-5.3 and DeepSeek V4 are closing in on the best, Kimi K3 is strong on agentics). And a point too often forgotten: an open-weight model performs much better in a good harness (a well-configured OpenCode) than in a raw chat.

But keep two limits in mind:

• The summit is still Claude. On the Quelle IA code ranking (in French), Claude Opus 5.5 (99.9) and Claude Fable 5.1 (98.6) lead. The best model that fits on a 128 GB mini-PC, Qwen 3.8 27B, scores 81.1 there (still a provisional score).
• The best open models don’t run on a mini-PC. Kimi K3 (2.8 T parameters), GLM-5.3 (744B), DeepSeek V4 Pro (1.6 T): that’s API or server. Even GLM-5.3-Flash, the best local model for code (88.9), needs two DGX Sparks. What fits (Qwen 3.8 27B, Gemma 4 26B A4B, gpt-oss-120b) is a notch below on long-haul agentic reliability and tool use.