Local LLM Hardware Matcher

About Local LLM Hardware Matcher

Want to run LLMs locally? First find out whether your GPU is up to it. Enter your VRAM and this tool instantly estimates every local model's needs at each quant tier (FP16 / Q8 / Q6 / Q4) using the params × bytes-per-param × 1.2 formula — bidirectionally: VRAM → which models run and the recommended tier; model → GB needed per tier and which common GPUs fit. Data comes from models.json's local-runnable models (Llama 4 Scout / Qwen3-32B / Gemma 3 27B). Results are conservative estimates — real usage also varies with context length and inference framework, as labelled. All math runs locally in your browser.

FAQ

What's the VRAM formula, and how accurate is it?

Estimate = params × bytes per param (FP16=2.0 / Q8≈1.06 / Q6≈0.82 / Q4≈0.5) × 1.2 (weights + ~20% runtime headroom), rounded to whole GB. It's deliberately conservative: Qwen3-32B at Q4 comes out at 19GB, matching community reports of 18–20GB. Real usage also depends on context length (KV cache grows with the conversation), the inference framework (llama.cpp vs vLLM memory strategies) and offload settings — budget more for long contexts.

Why does an MoE model like Llama 4 Scout need so much VRAM?

A mixture-of-experts model's "109B total / 17B active" means only 17B parameters fire per token — but ALL 109B weights must sit in VRAM/memory. Active params set the speed, not the footprint, so Q4 still costs ~65GB: think multi-GPU workstations or Mac unified memory. For single consumer GPUs see Qwen3-32B (24GB) and Gemma 3 27B (16GB).

Does insufficient VRAM mean I can't run it at all?

Not necessarily. Frameworks like llama.cpp support CPU/GPU hybrid inference (some layers in system RAM) and lower quants — you can still run it, just noticeably slower. This tool matches on the "fits fully in VRAM" standard and treats hybrid setups as out of scope. Apple Silicon's unified memory also works; treat available RAM as a rough equivalent of the VRAM budget.