Qwen3.8, Gemma 4 and gpt-oss: calculate KV-cache VRAM

Cloud Codes examines why a local LLM memory calculator can overestimate KV-cache requirements when it treats every layer as full attention. The walkthrough uses model configuration files to distinguish growing caches from fixed state and sliding windows. The speaker notes that the standard formula still applies to models with full attention in every layer.

For Qwen3.8-27B, the presenter counts 16 full-attention layers among 64 layers and calculates 62.5 MiB of cache per 1,000 tokens rather than 250 MiB. Gemma 4 12B combines a 1,024-token sliding window with periodic full attention. For gpt-oss-120b, the speaker calculates about 4.5 GiB of cache at 131K context, then explains why its native MXFP4 weights leave little room for further weight compression.

The video also separates cache size savings from accuracy and speed. It cites llama.cpp community tests in which Q8 changed few answers, while Q4 caused a large accuracy drop on one older Qwen model. Those results concern the tested models and settings. The reported failure was concentrated in quantized keys; quantizing values alone had much less effect.

GPU examples cover 8GB through 64GB tiers. Viewers planning to run models locally should distinguish weight size and cache estimates from total runtime memory: buffers also need space, and a smaller cache does not by itself establish that a model fits.