MLX-LM and LangGraph: build a local agent on Mac

This tutorial builds a local AI agent on an Apple Silicon Mac using MLX-LM and LangGraph. The speaker explains MLX's shared CPU and GPU memory model and lazy computation, then distinguishes that framework from MLX-LM, the package used to load and serve language models.

The setup serves Qwen-3-4B-Instruct-2507 in 4-bit through a localhost HTTP endpoint on port 8080. The first launch downloads the model if it is not already available. A Python application sends prompts to the server, while LangGraph manages state and connects the workflow's nodes.

Routing follows deterministic rules based on city and search arguments. It selects weather, search, both tools, or the model alone. Open-Meteo supplies geocoding and forecasts; the Wikipedia API supplies search context. The local LLM synthesizes the tool results, and a final node saves a Markdown report. Examples cover a Mysuru day trip and an umbrella question for Bengaluru.

The demo avoids a hosted LLM API key, but its weather and search calls still use external services. Local inference does not make the full workflow offline. The speaker reports that the small model worked for these examples and cautions that its reasoning may fall short of frontier models. The tutorial focuses on an inspectable workflow rather than a benchmark of model quality.