HomeAI ToolsLLM Inference Hardware Calculator

LLM Inference Hardware Calculator

March 6, 2025

Estimate VRAM & System RAM for single-user inference (Batch=1).
Model quant & KV cache quant are configured separately.

Model Configuration

Select Model:

Number of Parameters (Billions): 7 Model Quantization: Context Length (Tokens): 4096

Enable KV Cache

KV Cache Quantization:

System Type:

System Memory (GB): 128

Select GPU:

GPU VRAM (GB per GPU): 24 Number of GPUs: 1

VRAM Needed:

On-Disk Size:

GPU Config:

System RAM: