Pipecat

Pipecat is an open-source Python framework for building real-time voice and multimodal conversational AI agents. It was created by Daily, the WebRTC infrastructure company founded in 2016 and led by co-founder and CEO Kwindla Hultman Kramer, who open-sourced the project on Hacker News in May 2024. The framework solves a concrete engineering problem: orchestrating the speech-to-text, language model, and text-to-speech steps of a voice agent pipeline with low enough latency to feel like a natural conversation. It is BSD-2-Clause licensed, free to use, and requires no licensing fees.

At v1.1.0 (released April 2026), Pipecat supports over 100 integrations spanning 19+ STT providers (Deepgram, AssemblyAI, Whisper, Sarvam), 35+ TTS options (ElevenLabs, Cartesia, OpenAI, Smallest TTS), 25+ LLM providers (OpenAI, Anthropic, Mistral, xAI, Inworld Realtime), and transports including Daily WebRTC, LiveKit, WebSocket, Twilio, and WhatsApp. Client SDKs cover JavaScript, React, React Native, iOS (Swift), Android (Kotlin), C++, and even ESP32 for embedded devices. The ecosystem includes Pipecat Flows for structured conversation state, Pipecat Subagents for multi-agent handoffs, and the optional Pipecat Cloud managed hosting service from Daily for teams that want to skip self-managed infrastructure.

What Pipecat actually does in May 2026

Pipecat's pipeline model is its defining feature. Where managed platforms like Vapi execute the STT/LLM/TTS loop behind an abstraction layer, Pipecat makes every step explicit code. A developer connects a streaming transcription service, configures endpointing sensitivity, handles partial results, writes custom logic between transcription and the LLM step, and controls exactly how speech is synthesized and played back. This is the framework's core value proposition and its core cost: complete control with complete responsibility.

The framework reached stable v1.0.0 in April 2026, introducing a WebSocket-based OpenAI Responses API integration, async function call support, and Inworld Realtime LLM compatibility. A migration guide accompanied the release for teams upgrading from the 0.0.x pre-release series. v1.1.0 followed two weeks later, adding Mistral STT/TTS, DTMF (phone keypad) support, multilingual Deepgram modes, and xAI integrations. The project logged 11.7k GitHub stars by May 2026, with a consistent monthly release cadence throughout 2025-2026.

Pipecat is transport-agnostic by design. The same pipeline code runs over Daily's WebRTC infrastructure, LiveKit rooms, a raw WebSocket server, Twilio Media Streams, or local audio capture. The pipeline logic doesn't change when the transport changes. This makes it feasible to prototype locally, test with WebSockets, and deploy via Daily WebRTC or Twilio in production without rewriting core agent logic.

"The speech-to-speech models are the future. But today what you should do in production is a transcription model, a text-mode LLM, and a voice generation model." - Kwindla Hultman Kramer, CEO of Daily and creator of Pipecat, Freeplay AI interview, April 2026

Pipecat Flows extends the core framework with a state machine model for structured conversations: multi-step intake flows, branching on intent, and clean node transitions. Pipecat Subagents handles multi-agent architectures where a routing agent dispatches to specialized agents (billing, technical support, onboarding) and manages state handoffs. These are features that managed platforms either don't offer or implement in ways that are opaque to the developer.

Where Pipecat sits versus LiveKit Agents and Vapi

The voice AI framework category currently has three distinct architectural approaches, and Pipecat sits squarely in the open-source self-build lane alongside LiveKit Agents, differentiated from both by its philosophy and from managed platforms like Vapi by its fundamental model.

Pipecat vs. LiveKit Agents: Both are open-source frameworks that require developer infrastructure. The key mechanical difference is their relationship to the transport layer. LiveKit Agents is event-driven and tightly coupled to LiveKit's WebRTC SFU (Selective Forwarding Unit) architecture: agents join WebRTC rooms as participants, subscribe to audio tracks, and react to events. This coupling gives LiveKit Agents its exclusive capability: video support. Because LiveKit's SFU routes video streams between room participants, video agents with avatars (HeyGen, Tavus) can work within the native LiveKit room model. Pipecat's transport-agnostic pipeline model doesn't offer this natively. However, LiveKit's room model becomes a constraint for telephony-only or raw WebSocket deployments where the room abstraction adds overhead. Pipecat handles those with the same pipeline logic. Developer feedback broadly characterizes LiveKit as faster to get something working; Pipecat as offering more per-step control, at the cost of more tuning work on turn-taking and VAD.

Pipecat vs. Vapi: The comparison is really build vs. buy. Vapi is a closed-source managed platform where developers configure a voice agent via API parameters (system prompt, voice provider, tool definitions) and Vapi's infrastructure handles the rest. Time to first call is roughly two hours. The tradeoff is control: with Vapi, a developer cannot intercept or modify a transcription before it reaches the LLM, cannot tune VAD endpointing sensitivity, and cannot insert custom logic mid-conversation without hitting the managed model's constraints. Vapi costs $0.05-$0.13/minute all-in. Self-hosted Pipecat pays only for AI provider APIs, and above roughly 50,000 minutes per month, the cost savings are estimated at 80% versus Vapi. Below 10,000 minutes per month, the engineering overhead of running Pipecat often outweighs the per-minute savings.

"GPT-5, the latest Claude, Gemini 3, they all saturated what I thought was a really hard benchmark. But here's the thing: they're all too slow to use for a voice agent." - Kwindla Hultman Kramer, State of Voice AI interview, Coval.ai, 2026

For teams comparing Pipecat against Retell AI or Bland AI, the same build-vs-buy calculus applies: those managed platforms offer faster time-to-market for standard use cases, while Pipecat wins on customization depth, data privacy, and economics at scale.

What the pipeline reality looks like

A minimal Pipecat agent in production involves a few moving parts: a Python process running the pipeline, a transport connection (WebRTC or WebSocket), an STT provider streaming audio chunks and returning partial transcripts, a VAD (Voice Activity Detection) layer detecting when the user has finished speaking, an LLM inference call, a TTS provider streaming synthesized audio back, and playback logic that interrupts the bot's current speech if the user starts talking again (barge-in).

Getting each step right in production requires tuning. The framework's documentation includes an STT Latency Tuning guide specifically because this is a common stumbling block. GitHub issues document a 2-5 second gap between user speech ending and bot response beginning (issue #1694), a half-duplex gating bug where the audio output gate stays closed after the bot finishes speaking (preventing the next TTS playback until fresh user speech resets the state), and mid-sentence audio pauses with no obvious trigger (issue #2941). When a second participant joins a LiveKit-based session, severe latency and response queueing have been reported (issue #3218). A production guide published by a developer who deployed Pipecat at scale documented 26 distinct production issues spanning memory leaks, VAD misfires, and pipeline freezes.

Pipecat Cloud, Daily's optional managed hosting layer, offloads the infrastructure piece: provisioning agent containers, scaling concurrency, and handling SIP/PSTN telephony. Pricing is usage-based from $0.01/min for a 0.5 vCPU container up to $0.03/min for a 1.5 vCPU container, with no monthly minimums. Free Daily WebRTC transport is included for 1:1 voice sessions. Pipecat Cloud is positioned as the path for teams who want Pipecat's control over the pipeline logic but don't want to manage Kubernetes or Docker autoscaling for concurrent voice sessions. It launched in 2025 as what Daily described as "the first open source voice AI cloud."

For pairing recommendations, the community consensus in 2025-2026 for a technical team building a voice-only agent is Pipecat or LiveKit Agents plus Deepgram Nova-3 for STT and Cartesia Sonic-3 or ElevenLabs Flash v2.5 for TTS. The latency target is sub-800ms median for the full voice loop; the 500ms benchmark Kwindla demonstrated at Pipecat's 2024 launch using Deepgram + Groq + Deepgram Aura remains the reference point for what's achievable with well-tuned provider choices.

Who Pipecat is built for

Pipecat is written for engineers who need to ship custom voice agents and are willing to own the infrastructure and tuning required to make them production-ready. The sweet spot is teams with at least one Python-competent engineer, a use case that justifies the engineering investment (custom conversation logic, compliance requirements, scale economics above ~50K minutes/month, or a mix of all three), and a preference for vendor neutrality across AI service providers.

Customer support and call center automation are the primary production categories. A self-hosted Pipecat agent can slot into existing telephony infrastructure via Twilio or SIP, connect to a CRM via function calls, and handle thousands of concurrent inbound calls at a cost structure that managed platforms can't match at scale. Pipecat Subagents makes it practical to build routing logic that hands callers between specialized agents without losing conversation context.

Voice coaching, language learning, and education are the second major category. The Flows system handles structured lesson progressions. Multimodal extensions (Moondream for vision, HeyGen/Tavus for video avatars) are available for richer interactive tutoring experiences that go beyond audio-only exchanges.

Embedded and IoT voice agents represent a niche but real use case. Pipecat ships a C++ client SDK and explicit ESP32 support, covering developers building voice interfaces for hardware devices. This is a capability gap that neither Vapi nor Retell AI covers.

What Pipecat is not

Pipecat is not a no-code tool. There is no visual builder, no prompt-and-deploy interface, and no admin dashboard for non-engineers to configure agents. Every agent requires writing Python code for the pipeline and managing deployment infrastructure, whether self-hosted or via Pipecat Cloud.

It is not a JavaScript-first framework. The agent server runs Python 3.11+ with no official TypeScript runtime for the server side. Teams with TypeScript-only backends must run Pipecat as a separate Python service. The client SDKs (JavaScript, React, iOS, Android) handle the browser and mobile side, but the pipeline itself lives in Python.

It is not a good fit for small-volume projects where speed-to-market matters more than cost or control. If a team needs to ship a basic voice bot in 48 hours or runs under 10,000 minutes per month, the combination of Vapi's managed platform and 2-hour time-to-first-call will outperform the weeks of infrastructure setup that Pipecat requires. The framework's strength compounds at scale and depth of customization; below those thresholds, managed alternatives are the more practical choice.

It is also not yet a stable API in the traditional sense. v1.0.0 arrived in April 2026 with intentional breaking changes from the 0.0.x series and a required migration guide. Teams running pre-1.0 Pipecat in production had to upgrade their pipeline code to follow the new WebSocket-based architecture. The project is moving fast, which is both its appeal and its maintenance cost.