llmfit review: model recommendations on five machines

llmfit is a terminal tool for choosing a local LLM based on available hardware. The presenter explains how it detects CPU cores, RAM and GPUs, including multi-GPU setups, then recommends models and quantization levels. Its rankings combine quality, speed, fit and context, with different weights for chat and reasoning tasks.

The review examines recommendations on five machines: a 2012 Raspberry Pi, a Luckfox Pico Ultra W, a 2015 Intel MacBook Pro, an M2 Max MacBook Pro with 32 GB of unified memory, and an RTX 5090 desktop. The small boards expose serious limits. The presenter reports recommendations for roughly 400-billion-parameter DeepSeek R1 models despite their tiny memory capacities. For Falcon H1 Mini on the Pi, llmfit predicts 241 tokens per second, compared with about 0.3 in the presenter's earlier test.

Recommendations look more plausible on the laptops, though the presenter questions the Intel MacBook's speed estimates. Coding filters and context sorting change which models appear at the top. The video also explains bandwidth-based speed estimates, configurable assumptions and a community leaderboard for comparison with user benchmarks. These results make llmfit a starting point for people who want to run models locally, while showing why its estimates need checking against measured performance.