WhisperLiveKit

WhisperLiveKit is a self-hosted speech-to-text server for meeting transcripts, live captions and apps that need speech recognition as people talk. It processes audio on your own machine or server.

• Live transcription: Streaming recognition keeps context across audio segments, and one server can handle concurrent sessions in different languages.
• Speaker labels and translation: Optional speaker diarization distinguishes speakers, while translation produces text in a target language; availability depends on the backend and installed extras.
• Model choice: Supported backends include Whisper, Voxtral Mini, Qwen3-ASR, SenseVoiceSmall through FunASR and NVIDIA Canary-1b-v2 through NeMo.
• App integration: A streaming WebSocket API and compatible subsets of OpenAI and Deepgram APIs let developers connect existing clients; a bundled web UI and native macOS client are also available.
• Recognition context: Whisper-family and SimulStreaming backends accept names and terminology for individual sessions to guide transcription.

The project is open source under Apache 2.0 and supports Docker deployment with CPU or NVIDIA GPU processing. Qwen3-ASR streaming also runs on Apple Silicon through MPS, but requires an explicit language and provides estimated word timestamps. Canary runs on CPU or CUDA, with a GPU recommended for speed. SenseVoiceSmall supports Mandarin, Cantonese, English, Japanese and Korean transcription, but doesn't support direct English translation.