Colibrì vs llama.cpp: DeepSeek V4 CPU comparison

Can a desktop run a model larger than its RAM? This local LLM comparison tests Colibrì and llama.cpp with DeepSeek-V4-Flash-0731, a mixture-of-experts model with 284 billion total parameters and 13 billion active per token.

The speaker uses a Ryzen 5 5600X with 12 threads, 61 GB RAM, an RTX 3060 with 12 GB VRAM, and an NVMe drive. Colibrì receives the official 166.9 GB checkpoint; llama.cpp uses Unsloth's 161.9 GB UD-Q8_K_XL build. The speaker says Colibrì's DeepSeek engine has no GPU support, so the comparison includes a CPU-only llama.cpp run.

Colibrì works within a 56 GB memory budget and produces its first token after 25 seconds. The short run finishes in 69 seconds. llama.cpp initially fails while allocating a 147 GB repacking buffer. Disabling repacking with -nr (--no-repack), alongside keeping experts on the CPU, lets it run. Its reported total is 142 seconds, with generation around 0.9 tokens per second on CPU.

Longer prompts reverse the result. For a 3,000-token prompt, the speaker reports eight minutes for llama.cpp versus an hour and two minutes for Colibrì. He attributes the gap to llama.cpp's batched prompt processing and Colibrì's token-by-token approach. These measurements describe one machine and model setup, rather than a universal performance ranking.