Step-Audio 2

Step-Audio 2 is an audio AI model family for developers building voice assistants and applications that interpret recordings. It can reason about what people say and how they say it, as well as sounds beyond speech.

• Voice conversation: Generate spoken responses that take the conversation's context and the speaker's vocal cues into account.
• Audio understanding: Transcribe speech, describe audio, and answer questions about speech, music, and other sounds, including cues such as emotion, pitch, and speaking rhythm.
• Speech translation: Translate between English and Chinese with either text or spoken output; speech recognition also supports Cantonese, Japanese, and Arabic.
• Tools and retrieval: Connect conversations to external tools and retrieved text or audio, and adapt the output voice using retrieved speech.

The open-source releases are Step-Audio 2 mini, mini Base, and mini Think, licensed under Apache 2.0. They run locally through Python with PyTorch and CUDA, with a Gradio web demo for browser access. The vLLM backend supports streaming responses and deployment across multiple GPUs; the project also provides a Docker image. Local inference uses your own hardware and requires a CUDA-capable GPU.

The full Step-Audio 2 model is available through StepFun's hosted realtime console and AI Assistant mobile app. The console includes web search and requires a StepFun Open Platform API key. The mobile app includes web and audio search; these hosted services run through StepFun rather than your local model deployment.