MLX LM

MLX LM is a Python package for developers and researchers who want to generate text and adapt language models on Apple Silicon Macs. It uses MLX for local inference and training, with access through Python or a terminal.

• Text generation: Generate streaming responses or process batches of prompts, and use terminal chat that keeps context throughout a session.
• Model adaptation: Fine-tune models with low-rank or full model training, including support for quantized models.
• Model conversion: Convert and quantize supported models for local use, with optional uploads to the Hugging Face Hub.
• Long context: Reuse cached prompts across queries to reduce repeated processing; memory controls let you trade prompt processing speed or output quality for lower RAM use.
• Distributed workloads: Run inference and fine-tuning across machines through MLX's distributed capabilities.

The package is open source under the MIT license. It works with supported models from the Hugging Face Hub, including Mistral, and with quantized models from the MLX Community. Inference and fine-tuning run on your hardware. Model downloads and optional uploads use Hugging Face; its separate model conversion Space runs as a hosted service.

Available RAM affects which models are practical to run. Models that are large relative to your Mac's memory can be slow, and the memory optimization for large models requires macOS 15 or later.