llama.cpp: CPU and H100 tests with Kimi, GLM and MiMo

In this Russian-language benchmark, ServerFlow compares Kimi K2.7 Code, GLM-5.2 and Xiaomi MiMo V2.5 Pro on a 32-core AMD EPYC server with 1 TB of DDR4 RAM and one NVIDIA H100 PCIe. llama.cpp divides inference work between system memory, CPU and GPU.

The team reports GLM-5.2 Q6_K_XL generation at 2.95 tokens per second on CPU and 5.93 with the H100. Q4_K_XL reaches 8.49. Increasing ubatch raises reported prompt processing from 13 to 146 tokens per second for Q6; Q4 reaches 250. The speaker attributes generation limits mainly to RAM bandwidth and prompt processing limits to PCIe Gen3. MiMo reaches 8.78 tokens per second, while Kimi reaches 9.81 after switching from IQ4_XS to a Q4_K quantization. These are results from one configuration.

An OpenCode task asks the models to build a game with API-controlled worms. The team reports a 51-minute Kimi run, about 40,000 input and output tokens, and roughly hour-long completion times across the models.

Their cost comparison estimates at least 6.5 million rubles for core hardware and 181 rubles of electricity per million output tokens. Electricity alone looks cheaper than the cited Kimi API price, but capital costs overwhelm that saving in their calculation. This is a worked estimate, rather than a current price or payback guarantee.