Kimi K3 and DeltaFin: laptop inference explained

The speaker explains how DeltaFin runs Kimi K3 on a 2021 M1 Max laptop with 64 GB of RAM by reading weights from storage rather than holding the whole model in memory. The reported steady decode speed is 14.6 seconds per token. The maintainer describes the project as a research artifact and a test bed for streaming inference, rather than a practical chat setup.

Kimi K3 is described as a mixture-of-experts model with 2.8 trillion total parameters and 104 billion active per token. The architecture discussion covers its million-token context window and recurrent attention state. The speaker also explains MXFP4 quantization-aware training and Unsloth's smaller quantizations, including a 594 GB build with reduced agreement on next-token choices.

DeltaFin keeps a quantized model spine separate from routed experts and reads the selected experts for each token. According to the walkthrough, that requires about 25.8 GB of reads per token. A full local install needs roughly 1.7 TB of disk space; a smaller cache can stream weights over the network, with a reported delay of about three minutes per token.

For readers exploring a local LLM, the limitations matter: DeltaFin uses greedy decoding, ignores temperature and top-p settings, and accepts one request at a time. Long prompts also cost more because prefill touches more experts.