DeepInfra

DeepInfra is a cloud inference platform built specifically for running open-source AI models cheaply at high volume. It is not a model maker but an inference host: you send requests to its OpenAI-compatible endpoints, the models run on DeepInfra's own vertically integrated GPU stack, and results come back in milliseconds. It was founded in September 2022 in Palo Alto by Nikola Borisov, Yessen Kanapin, and Georgios Papoutsis, the same team that built the messaging app imo used by more than 200 million people. By mid-2026 the platform processes close to five trillion tokens per week across eight US data centers, which is real production scale rather than a launch metric.

A single API key reaches more than 150 models, covering text generation, reasoning, embeddings, image, speech-to-text, and more, from DeepSeek, Llama, Qwen, Mistral, Gemma, and others. Switching is a base-URL change for anyone already using the OpenAI SDK, LiteLLM, or similar tooling. The draw is price: Mistral Nemo at $0.02 input and $0.04 output per million tokens is near the cheapest available anywhere, and a dedicated H100 runs $1.79 per hour, a fraction of premium competitors. DeepInfra carries SOC 2 and ISO 27001 with a zero-retention policy, and its May 2026 Series B included NVIDIA as a strategic investor.

What DeepInfra runs in June 2026

The product is deliberately narrow and deep: serverless inference over a curated 150-plus model catalog, billed purely per use, with no subscriptions or seat licenses. Text generation is the core, but the catalog also spans embeddings, image models like FLUX, speech-to-text, and rerankers, so a multimodal app can stay on one vendor. For teams that need isolation, DeepCluster offers dedicated GPU instances with private endpoints and support for custom or fine-tuned weights, though there is no managed fine-tuning pipeline. The company's trajectory is steep: a $18 million Series A in April 2025 disclosed token volume up 8,000x since seed, and on May 4, 2026 a $107 million Series B co-led by 500 Global with NVIDIA as a named investor came alongside collaboration on Nemotron inference optimization and access to Blackwell-class GPUs. That NVIDIA relationship is the clearest signal that DeepInfra is more than a cheap-API novelty.

DeepInfra versus Groq and Together AI

Against Groq, the trade is cost versus speed. Groq's custom LPU silicon serves a 120B model at roughly 476 tokens per second against DeepInfra's 161, but DeepInfra charges a fraction of the price and carries ten times the model selection, since Groq runs only a handful of models on inference-only hardware. Pick Groq when every hundred milliseconds matters, DeepInfra when cost and breadth do. Against Together AI, DeepInfra is simply cheaper, often 67 to 76% cheaper on the same Llama models, but Together has the clear advantage in fine-tuning, offering a full train-and-serve pipeline that DeepInfra lacks, plus a deeper mixture-of-experts catalog. A team that needs to fine-tune and deploy from one API leans Together; a team optimizing pure inference cost leans DeepInfra. On reliability, routing data from aggregators like OpenRouter has flagged weaker uptime on some of DeepInfra's less-popular endpoints, which is the recurring caveat.

Where DeepInfra frustrates developers

The loudest complaint is price stability. Because models are priced individually and can change without much warning, a workload's economics can shift under you.

"They lure you in with cheapness and they increase the price by 400% or delete the model forcing you to use more expensive versions." Simon M., SourceForge, September 2025.

That reviewer documented three pricing changes in under a month, including a deleted model with no replacement. The second pattern is performance on niche models: popular endpoints stay fast, but less-used ones can degrade badly under load, and bug reports do not always get a response.

"Deepinfra slow to an absolute crawl. I really liked Deepinfra but something doesn't seem right." wolttam, Hacker News, April 2026.

Support runs mostly through a sales form rather than a technical channel, model deprecations arrive without migration paths, and the prepaid billing tiers quietly raise the minimum top-up as you scale, up to $10,000 at the top tier. None of this is disqualifying at the price, but it is the reason production teams keep a fallback provider configured.

Who should use DeepInfra, and who should not

DeepInfra is an excellent fit for cost-sensitive teams processing high token volumes, for developers who already use the OpenAI SDK and want cheaper open models with a base-URL change, for multimodal products that want a wide catalog from one vendor, and for buyers who need SOC 2 or ISO 27001 with zero data retention. Funded startups can also tap the DeepStart program for a billion free tokens.

It is the wrong choice when you cannot tolerate sudden pricing or model changes without notice, when you need a managed fine-tuning pipeline (Together AI fits better), when you need guaranteed structured-output reliability on every model (Fireworks AI is stronger there), or when latency is the product and Groq's hardware advantage is worth the premium. Anyone building on a brand-new or niche model should test reliability at real load before committing.