大模型显存数学公式与硬件底层规范

Total VRAM is calculated as: Total_VRAM = Weights_GB + KV_Cache_GB + Activation_GB + CUDA_Overhead_GB. For a Q4_K_M quantized model, Weights_GB ≈ Parameters_B × 0.58 × 1.05. Dynamic KV Cache depends on attention architecture (GQA vs MLA) and scales with context length and batch size. OS and CUDA context runtime overhead typically requires 1.0GB to 1.4GB on Windows/Linux.

Standard GQA caches separate uncompressed key and value vectors of dimension d_head per KV head per layer (2 × n_layers × n_kv_heads × d_head × context_len × precision). In contrast, DeepSeek MLA low-rank projects the joint KV state into a single compressed latent vector of dimension d_c = 512 plus decoupled positional RoPE keys d_rope = 64 (576 scalars per token per layer), achieving up to 82% VRAM reduction at 32k to 128k context lengths.

During single-batch token generation, each token requires every active layer weight to be fetched. Dedicated GDDR6X/GDDR7 VRAM provides 1,008 to 1,792 GB/s bandwidth. A PCIe 4.0 x16 bus provides only 31.5 GB/s (less than 2% of VRAM bandwidth). When layers are offloaded to system DDR RAM, weight streaming across PCIe creates an insurmountable pipeline stall, dropping generation speed from 30+ tok/s down to ~1.2 tok/s.

Traditional contiguous memory allocation causes up to 60% memory waste due to internal and external fragmentation when sequence lengths vary dynamically. PagedAttention (as pioneered in vLLM) partitions the KV cache into fixed-size virtual memory blocks (pages), reducing memory waste to less than 4% and enabling significantly larger batch sizes and context windows in dedicated VRAM.