Deltafin: Kimi K3 local setup and 64 GB RAM limits

Deltafin is presented as an experiment in running Kimi K3, described by the speaker as a 2.8 trillion parameter model, on a single workstation. The walkthrough reviews repository instructions and reported results from an Apple M1 Max with 64 GB RAM. It gives an overview of local LLM setup, rather than documenting a fresh installation or test.

The setup outline covers cloning the repository, creating a virtual environment and downloading model data. The speaker lists a C/C++ compiler, Python and PyTorch as requirements, and says the project has been tested on Apple silicon macOS and Linux. Windows is not presented as a tested platform. Reported disk requirements are about 1.7 TB for the full model or 215 GB for streaming.

The architecture reads model components layer by layer from storage. For each token, the router selects 16 experts per layer; Deltafin reads these locally or fetches them from Hugging Face on demand. The walkthrough also mentions an optional int8 spine and an API-compatible server.

Speed is the main limitation. The speaker cites about 0.0687 tokens per second, a median of 14.6 seconds per token, and more than three minutes for uncached data. Later benchmark figures differ, so these numbers should not be treated as one consistent performance result. The speaker expects newer hardware to improve speed but supplies no measurements from it.