Sonar (formerly Aphrodite Engine)

Sonar serves language and multimodal models for developers and teams running AI applications on their own hardware. It builds on vLLM and works with Hugging Face-compatible models.

• Application APIs: Connect apps through OpenAI-compatible endpoints with streaming, embeddings, tool calls and reasoning output, or use the Python API for local and batched inference.
• Concurrent inference: Continuous batching and prefix caching help process requests together and reuse shared prompt work; quantized weights reduce memory needs, while speculative decoding can speed up generation.
• Multimodal and structured output: Serve image, audio and video models, request structured responses, and serve LoRA adapters alongside models.
• Additional interfaces: Use Anthropic and Kobold APIs, plus endpoints for scoring, reranking and transcription.
• Distributed serving: Spread inference across GPUs or multiple nodes and monitor servers with Prometheus metrics and health endpoints.

Sonar is open source under AGPL-3.0. It runs on Linux and Apple silicon macOS with Metal, with Docker and WSL 2 deployment options. Supported hardware includes CPUs, NVIDIA CUDA, AMD ROCm, Intel XPU and Google TPU. Model and feature support depends on the device, data type and quantization method, so multimodal inference and other capabilities aren't available in every combination.