Deepgram

Deepgram is an AI speech platform built by Deepgram, Inc., a San Francisco company founded in 2015. It offers three production APIs: speech-to-text using the Nova-3 model, text-to-speech using Aura-2, and a Voice Agent API that wires STT, TTS, and LLM orchestration into a single managed WebSocket pipeline. The platform has no consumer interface. You send audio or text via API and get structured transcripts, synthesized speech, or full voice agent sessions back. Over 450 enterprise customers use Deepgram in production, including Twilio, Aircall, and Jack in the Box.

Nova-3, launched February 12, 2025, is the current flagship STT model. It achieves a median word error rate of 6.84% on streaming audio, a 54.3% improvement over the next closest managed API competitor. It supports real-time code-switching across 10 languages simultaneously, offers keyterm prompting for instant vocabulary adaptation without model retraining, and operates at sub-300ms latency for streaming sessions. The Voice Agent API reached general availability on June 16, 2025, benchmarked at 71.5 on the Voice Agent Quality Index, outperforming OpenAI by 6.4% and ElevenLabs by 29.3% on a composite of latency, interruption rate, and response coverage. Aura-2 TTS delivers sub-200ms time-to-first-byte with 40+ English voices across 7 languages.

What Deepgram actually does in April 2026

Deepgram's core product is managed speech infrastructure. You connect to its API via WebSocket for real-time streaming or via HTTP for batch processing of pre-recorded files. Nova-3 handles the heavy lifting on the STT side: it accepts audio streams and returns word-level transcripts with timestamps, optional speaker diarization, punctuation, and real-time redaction of up to 50 entity types (phone numbers, credit card numbers, social security numbers).

The keyterm prompting feature deserves specific attention. Rather than requiring model fine-tuning or custom vocabulary uploads, Nova-3 accepts up to 100 domain-specific terms per API request and boosts their recognition probability without any retraining cycle. This makes it faster to adapt Deepgram for medical, legal, or technical audio than traditional fine-tuning pipelines allow. The 100-term cap is a real constraint for large vocabularies, but for most production use cases, it covers the highest-priority terminology.

The Voice Agent API abstracts the entire voice agent pipeline. Developers previously had to stitch together a separate STT service, a VAD layer for barge-in detection, an LLM API, and a TTS endpoint. Deepgram's Voice Agent API handles all of this through a single WebSocket session: it manages turn-taking, interrupt detection, session state, and real-time prompt updates. You can bring your own LLM while using Deepgram's STT and TTS, or use the full Deepgram stack at $4.50 per hour.

Deployment options include public cloud, private VPC inside AWS or Azure accounts, and on-premises via Docker or Kubernetes containers. The on-premises path matters for healthcare and financial teams: Deepgram's containerized architecture delivers identical latency and accuracy to the cloud while keeping audio data behind corporate firewalls. This is available on Enterprise contracts only.

"The most accurate real-time STT we've tested.. the first speech-to-text model to combine real-time multilingual transcription, live vocabulary injection, and sub-300ms latency." - Stephen FIYINFOLUWA Oladele, Neurl Creators, Substack review, 2025

Where Deepgram sits versus AssemblyAI and Whisper

The comparison that matters most for production decisions is Deepgram Nova-3 against AssemblyAI Universal-2/Slam-1 and OpenAI Whisper. These three cover most of the developer STT market, but they make fundamentally different engineering tradeoffs.

Deepgram vs. AssemblyAI: AssemblyAI's Slam-1 model combines an LLM decoder with a traditional ASR encoder, which gives it more flexible prompt-based customization and handles up to 1,000 domain-specific terms per request, versus Nova-3's 100-term cap. AssemblyAI Universal-2 supports 100+ languages compared to Nova-3's narrower but more accurate multilingual set. AssemblyAI also has deeper out-of-the-box audio intelligence: topic detection, entity detection, chapter generation, and its LeMUR audio LLM for conversational queries over transcripts. Deepgram's audio intelligence (summarization, sentiment) is simpler and has known hallucination issues on dense technical speech. Deepgram wins on raw streaming latency (sub-300ms vs. AssemblyAI's 300-600ms range), on-premises deployment (AssemblyAI is cloud-only SaaS), and pricing at volume. AssemblyAI Universal-2 is $0.37/hr; Nova-3 sits at $0.46/hr on Pay-As-You-Go but drops to $0.39/hr on the Growth plan.

Deepgram vs. Whisper: Whisper is an open-source encoder-decoder Transformer from OpenAI, trained on 680,000 hours of multilingual audio, with weights available for self-hosting. It covers 57 languages and is free at the model level. But Whisper does not natively support real-time streaming: getting it to work for live audio requires workarounds like whisper.cpp, adding meaningful engineering overhead. Whisper large-v3 takes 10-30 minutes to transcribe an hour of audio depending on hardware; Deepgram transcribes the same hour in roughly 20 seconds. Self-hosting Whisper on a GPU instance (typically $0.50-$1.50/hr for an A10G equivalent) brings the compute cost close to Deepgram's $0.46/hr, making the managed-vs-self-hosted tradeoff less clear-cut than the "Whisper is free" framing suggests.

"I chose Deepgram specifically because it advertised itself as being the fastest to minimize response delays." - Alyx1337, Hacker News, December 2023

What the voice agent pipeline reality looks like

The Voice Agent API is the product Deepgram is most actively positioning as a differentiated moat. Building a voice agent before this API existed meant running four separate services: an STT endpoint for transcription, a VAD layer for barge-in detection, an LLM API for response generation, and a TTS endpoint for speaking the response. Each hop adds latency, and stitching error states across four services means more failure modes to handle in production.

Deepgram's Voice Agent API collapses this into a single WebSocket session. You send audio; the API handles VAD, STT, LLM routing (Deepgram's default or your own), TTS synthesis, and response audio streaming back. Real-time prompt injection lets you update the system prompt mid-session, and model hot-swapping allows changing TTS voices during a live call. Jack in the Box is using this architecture for drive-through ordering. Aircall and OpenPhone are integrating it for call handling.

WebSocket connection management, however, is where developers consistently hit friction. Deepgram's GitHub discussions show an ongoing pattern of CLIENT_MESSAGE_TIMEOUT errors when audio starts streaming before the server has confirmed the SettingsApplied message. The correct handshake requires waiting for Welcome, then Settings, then SettingsApplied confirmation before sending any audio data. Misconfiguring this drops the connection within 2-3 seconds. The fix is documented but not prominently surfaced in the quickstart guides.

The default concurrent streaming quota is capped at 1,200 requests. Scaling beyond that requires contacting Deepgram's sales team, which introduces friction for fast-growing teams who hit the limit unexpectedly in production.

Who Deepgram is built for

Deepgram's sweet spot is engineering teams building production voice infrastructure where latency under 300ms is a requirement, not a preference. Real-time customer support analytics, IVR systems, voice-enabled SaaS features, and conversational AI agents all fit this profile well. Contact center platforms processing thousands of concurrent audio streams benefit from the Nova-3 streaming architecture and the on-premises deployment option for regulated industries.

The $200 free credit (no credit card required) covers roughly 433 hours of Nova-3 Monolingual transcription at Pay-As-You-Go rates. This is a substantial exploration budget for solo developers and small teams evaluating the API before committing to any plan.

What Deepgram is not

Deepgram is an API. There is no web interface for uploading an audio file and getting a transcript back. Non-technical users who want to transcribe podcast recordings or meeting audio without writing code should look elsewhere: Otter.ai, Rev, and Descript all offer UI-based transcription workflows. Deepgram is for developers integrating speech into an application.

Custom model fine-tuning is not self-serve. If 100 keyterms per request is insufficient and you need to train a domain-specific model on your own audio data, that requires an Enterprise contract. Teams building for 100+ languages should check AssemblyAI's Universal-2 instead. Teams whose primary requirement is audio intelligence depth, specifically conversational queries over transcripts, topic detection, or entity extraction, will find AssemblyAI's LeMUR and Slam-1 substantially more capable.

Skip Deepgram if you need a zero-infrastructure-cost path and have the engineering capacity to self-host. Whisper's weights are free, and running inference on your own hardware eliminates per-minute fees entirely, at the cost of DevOps overhead and the complexity of building real-time streaming support from scratch.