Pipecat

Pipecat is an open-source Python framework for developers who want to run voice and video agent applications locally or on their own servers. It hosts the conversation pipeline, while speech and model processing depend on the services you choose. Integrations such as Soniox, OpenAI and Cartesia send the relevant requests to those providers.

Its main focus is the timing of live conversations. Audio, video, text and images move through a shared pipeline, and an agent can begin responding while processing continues. Interruption handling, turn detection and context management help it deal with pauses and overlapping speech.

Developers can swap speech recognition, language and vision services without replacing the whole application. WebRTC and WebSockets connect agents to applications, while SIP and PSTN support phone conversations. Client SDKs and a UI kit cover the application side.

For longer tasks, Pipecat can hand work to specialist agents or run independent tasks in parallel, including across processes and machines. Pipecat Flows supports conversations that need a defined path. Evaluations, metrics and OpenTelemetry traces let teams inspect behavior and diagnose problems.

The self-hosted framework uses the BSD 2-Clause license and runs wherever Python runs. Pipecat Cloud is a separate managed offering that hosts, scales and monitors agents; Pipecat Enterprise deploys that cloud service in a customer's AWS, Azure or GCP environment.