sherpa-onnx

An open-source speech toolkit that runs ONNX models locally for transcription, speech synthesis and speaker analysis on desktop, mobile and embedded devices.

Screenshot of sherpa-onnx website

sherpa-onnx is a toolkit for developers building speech features that work offline on their own devices. It covers transcription, speech generation and audio analysis, with processing kept local.

  • Speech recognition: Transcribe live audio or recordings with models including Whisper, Zipformer, Paraformer and SenseVoice, with support for adding punctuation.
  • Speech synthesis: Generate speech with Kokoro, Piper or Matcha, and use Pocket or ZipVoice for voice cloning.
  • Speaker analysis: Separate a recording by speaker, identify or verify speakers, and detect the spoken language.
  • Audio processing: Detect speech and keywords, label audio events, reduce speech noise and separate audio sources.

The toolkit uses ONNX models through ONNX Runtime rather than PyTorch for inference. It runs on Linux, macOS and Windows, as well as Android, iOS, HarmonyOS and embedded devices such as Raspberry Pi and RISC-V systems. CPU execution is supported, with NVIDIA GPU support on Linux and Windows and NPU support for Rockchip, Qualcomm, Ascend, Axera and Intel hardware.

Bindings include Python, JavaScript, Swift, Rust and other languages, alongside Flutter and Tauri support. WebAssembly supports browser execution, and WebSocket servers allow self-hosted speech services. Separate Hugging Face Spaces provide hosted demos. The code is open source under Apache 2.0.

Similar to sherpa-onnx