DeepSeek-V3 671B with Ollama: CPU setup and speed comparison

Uploaded on February 9, 2025, this Russian-language tutorial demonstrates DeepSeek-V3 with 671 billion parameters in Q4 quantization through Ollama. The setup and performance figures describe that demonstration; the source does not specify an Ollama version.

The speaker uses a 48-core AMD EPYC server with 512 GB of DDR4 RAM and disables its four RTX 3090 Ti cards for CPU inference. He chooses this setup because the quantized weights fit in system memory, while assembling enough GPU memory would be more difficult. Ollama lists the weights at 404 GB, and the demonstration shows roughly 416 to 417 GB of RAM in use after loading.

The walkthrough covers connecting to the server over SSH, starting the model and waiting for weights to load from disk. The speaker says the initial download took several hours. His local LLM benchmarks separate prompt processing from answer generation: he reports about 160 prompt tokens per second for one request and roughly four generated tokens per second, with variation across worker counts.

He compares the result with the official DeepSeek chat and Fireworks, which respond faster in his examples. He also starts downloading the full DeepSeek-R1 model and discusses smaller distilled models for GPU use. His expectation that full R1 would match V3's speed is not a measured result in this demonstration. The provider speeds are observations from the video, not current service guarantees.