oMLX setup on Mac for Claude Code and Cursor

The presenter describes oMLX as an open source server for running a local LLM on Apple Silicon. The presenter explains its approach to a coding assistant that slows down as conversation history grows: reuse matching KV cache blocks instead of processing the same prefix again. Frequently used blocks stay in memory, while a second tier stores blocks on SSD for later reuse, including after server restarts.

The claimed benefit is shorter time to first token during long sessions with repeated prefixes. It does not make an individual model generate tokens faster. The presenter says cold starts and one-off chats see little benefit, and reordered system prompts or tool schemas can prevent cache reuse.

The setup section lists macOS 15 Sequoia or later, Python 3.11 through 3.13, and M1 through M4 chips. It describes the signed DMG setup and mentions Homebrew and source installation, whose commands are deferred to the description. The presenter recommends the DMG for its prebuilt custom kernels and says building those kernels requires full Xcode. Existing Hugging Face caches and LM Studio model folders can supply models.

Claude Code connects through an Anthropic-compatible endpoint; Cursor uses an overridden OpenAI base URL. The walkthrough also covers CLI configuration for other agents. Performance figures come from project benchmarks on an M3 Ultra with 512 GB and a separate developer account, rather than a controlled test by the presenter. The suggested check is whether cache blocks accumulate in the admin dashboard across turns.