Moshi

A local voice AI model and dialogue framework that listens while speaking, with MLX support for Mac and iPhone and self-hosted server backends.

Moshi is a voice AI model and dialogue framework for researchers and developers building spoken applications. It processes speech directly and can listen while it speaks, including information such as emotion and non-verbal cues.

Simultaneous conversation: Separate audio streams for the user and the model allow speech to overlap without requiring strict turn-taking.
Streaming speech: The Mimi audio codec handles incoming and outgoing sound, while Moshi predicts text alongside its spoken responses.
Voice models: Moshiko provides a male synthetic voice, and Moshika provides a female synthetic voice.
Local and server backends: Use PyTorch for research, MLX for on-device inference on Mac and iPhone, or Rust for production servers.

A browser interface or command-line client can connect to a locally running model. Kyutai also offers a hosted online demo, separate from inference on your hardware.

The PyTorch implementation requires a GPU with 24 GB of memory. MLX supports quantized models, and the Rust backend supports CUDA GPUs and Metal on macOS. Windows isn't officially supported. The code is open source under Apache 2.0; model weights use CC-BY 4.0.