llmfit tutorial: find models that fit your hardware

llmfit helps choose a local LLM before downloading its weights. The tutorial describes a Rust terminal tool that detects RAM, CPU and GPU hardware, then ranks models by suitability. Its table shows fit ratings, estimated generation speed and a recommended quantization. A model that fits in memory may still be too slow for comfortable chat, so the speaker advises checking both fit and speed.

The scoring considers quality, speed, memory fit and context. llmfit selects the highest-quality quantization that fits the available memory and can try a shorter context when necessary. The explanation also covers KV cache growth during longer conversations and how the tool assesses mixture-of-experts models.

For people planning to run models locally, Plan mode estimates the hardware needed for a chosen model and compares it with the current machine. Hardware Simulation recalculates results for a hypothetical configuration, with an RTX 5090 as one example. The tutorial describes provider support for Ollama, llama.cpp, MLX, Docker Model Runner and LM Studio; MLX appears only on Apple Silicon.

The speaker reports a 30% error in speed estimates and explains how users can contribute measured results through browser-based GitHub authentication. The closing setup section covers installation, JSON output for scripts and an HTTP API for checking multiple machines.