LLM VRAM calculator
A local LLM needs memory for its weights, a KV cache that grows with context and users, and runtime overhead. Choose a model and settings to see the total and an indicative hardware fit.
Estimated memory
26.1 GB
- Weights
- 19.9 GB
- KV cache
- 4.3 GB
- Overhead
- 2.0 GB
Indicative hardware fit
1× AMD Radeon AI PRO R9700
32 GB total · Workstation · ROCm. Needs 26.1 of 28.8 GB usable (90% of memory).
Check that your runtime supports ROCm for this model.
Prefer CUDA? 1× NVIDIA RTX PRO 6000 Blackwell Workstation Edition also fits.
View this optionIndicative memory estimate, not a performance guarantee. Real usage varies by runtime, batch settings and paged attention. Speed is not estimated here.
How the estimate is calculated
Architecture figures are read from each model’s published config, last verified 2026-10-27. Sliding-window models are sized at full context on every layer, so their estimate is conservative.
Weights
Parameters × bits per weight ÷ 8. A 70B model at 4-bit is about 43 GB; at FP16 about 141 GB.
KV cache
2 × layers × KV heads × head size × context tokens × concurrent users × bytes per value. Models with few KV heads (grouped-query attention) need far less than a parameter-count rule of thumb suggests.
Overhead
1 GB for the runtime context plus 4% of weights and cache for compute buffers. Each extra GPU in a split adds another 1 GB.
Hardware fit
The smallest configuration in our catalogue whose usable memory (90% of total) covers the estimate. It is a starting point; the advisor and a specialist validate it.
Formats and models in the calculator
| Weight format | Bits per weight | Note |
|---|---|---|
| FP16 / BF16 | 16 | Full precision, as released |
| 8-bit (Q8_0 / FP8) | 8.5 | Near-lossless for most models |
| 6-bit (Q6_K) | 6.56 | Very close to 8-bit quality |
| 5-bit (Q5_K_M) | 5.69 | Good balance |
| 4-bit (Q4_K_M) | 4.85 | The common local default |
| 4-bit MXFP4 | 4.25 | gpt-oss native format |
| 3-bit (Q3_K_M) | 3.91 | Noticeable quality loss |
| FP16 cache | 16 | KV cache, per value |
| 8-bit cache | 8.5 | KV cache, per value |
| 4-bit cache | 4.5 | KV cache, per value |
| Model | Params | Layers | KV heads | Source |
|---|---|---|---|---|
| Qwen3 8B | 8.2B | 36 | 8 | Config |
| DeepSeek-R1 Distill Llama 8B | 8B | 32 | 8 | Config |
| Qwen3 14B | 14.8B | 40 | 8 | Config |
| gpt-oss 20B | 21B | 24 | 8 | Config |
| Mistral Small 24B (2501) | 23.6B | 40 | 8 | Config |
| Qwen3 30B-A3B (MoE) | 30.5B | 48 | 4 | Config |
| Qwen3 32B | 32.8B | 64 | 8 | Config |
| DeepSeek-R1 Distill Llama 70B | 70.6B | 80 | 8 | Config |
| Qwen2.5 72B | 72.7B | 80 | 8 | Config |
| gpt-oss 120B | 117B | 36 | 8 | Config |
Questions about the estimate
Why is my estimate higher than the model file size?
The file holds only the weights. Running the model also needs the KV cache, which grows with context length and with every concurrent user, plus runtime buffers.
Why does the KV cache matter so much?
Each token in the context stores a key and a value for every layer and KV head. Long contexts or many users can make the cache larger than the weights, which is why we compute it from the model's own architecture.
Does it fit if the number is just under my GPU's memory?
Keep about 10% free. Drivers, fragmentation and runtime spikes use memory too, so the hardware suggestion only counts 90% of each card as usable.
Can I split a model across two GPUs?
Yes, with runtimes that support tensor or pipeline parallelism, such as vLLM or llama.cpp. Each GPU adds some overhead, and speed depends on the interconnect.
Does this tell me how fast the model will run?
No. This sizes memory only. Throughput depends on the GPU, the runtime, batching and the model, and we only publish speed figures we have measured.