Baseten

Baseten is a managed inference platform for putting custom and open-source AI models into production as fast, autoscaling APIs. It was founded in 2019 in San Francisco by four engineers who met building machine-learning fraud systems at Gumroad, with Tuhin Srivastava as CEO, and its entire pitch is captured in its tagline, "inference is everything." You package a model, and Baseten runs it on reserved GPUs with autoscaling, monitoring, logging, and compliance handled for you, collapsing the months of infrastructure work that normally sit between a trained model and a production endpoint. Unlike a pure model-API reseller, Baseten is built around your models, including fine-tuned and fully proprietary ones, with self-hosted and in-VPC options for teams that cannot send data to a shared cloud.

The product has four main pieces: Dedicated Deployments, which are reserved autoscaling GPUs billed by the minute; Model APIs, which give per-token access to popular open models like DeepSeek and Llama; Baseten Chains, which run multi-model pipelines with each step on purpose-matched hardware; and Baseten Loops, a 2026 SDK for reinforcement learning and fine-tuning. The on-ramp is Truss, Baseten's open-source model-packaging framework. Production customers include Cursor, Notion, the medical-AI company Abridge, and the legal-AI company Harvey, and the platform holds SOC 2 Type II and HIPAA compliance. NVIDIA invested directly in Baseten's January 2026 funding round, a signal of where the GPU maker sees inference demand heading.

What you actually deploy on Baseten in June 2026

The core unit is a model deployment. You wrap a model with Truss, push it, and Baseten serves it on GPUs ranging from a modest T4 up to a B200, scaling replicas up and down with traffic and even to zero when idle. For teams that do not want to package anything, Model APIs offer instant per-token access to common open models, while Chains lets you stitch several models together, say speech-to-text feeding an LLM feeding a summarizer, with each stage on the right hardware. Loops, added in 2026, closes the gap from training to serving so a fine-tuned checkpoint can be promoted to a live endpoint with one command.

The momentum behind the platform is hard to overstate. On January 23, 2026, Baseten raised $300 million at a $5 billion valuation with NVIDIA among the investors, and independent estimates show revenue tripling from roughly $200 million to $600 million annualized between December 2025 and March 2026, on the back of inference volume growing about 100x year over year. This is a company being pulled forward by demand rather than hunting for it.

Baseten versus Modal and Replicate, mechanically

Modal and Baseten solve overlapping problems from opposite ends. Modal gives you serverless Python functions defined with decorators and bills per second of execution, which is ideal for bursty batch jobs and general compute you build yourself. Baseten is model-as-a-service: the deployment, not the function, is the unit, and inference optimizations like kernel tuning and weight caching come built in. Baseten's delivery network can bring a sub-20GB model online in under ten seconds, but a warm replica bills continuously, whereas Modal's per-second model rewards bursty traffic. On raw hardware, Modal's H100 runs around $3.95 per hour against Baseten's $6.50, so you pay Baseten for the managed serving layer. Replicate is the other reference point, though it was acquired by Cloudflare in early 2026 and folded toward Workers AI. It always targeted individual developers prototyping against a huge community model catalog rather than enterprises deploying proprietary models under compliance, which is squarely Baseten's territory.

What replica-based billing really costs

Dedicated Deployments bill per minute of active replica time, which works out to roughly $0.63 per hour for a T4, $4.00 for an A100, $6.50 for an H100, and $9.98 for a B200. There is no charge for idle time, but the important word is replica, because each running replica costs money continuously, so a two-replica setup for redundancy doubles the bill immediately. Model APIs are billed per token instead, with DeepSeek V4 at $1.74 input and $3.48 output per million, which is simpler for standard models. New accounts get free credits, and qualifying seed-to-Series-A startups can claim up to $25,000 through the startup program.

"Baseten plays a central role in our AI infrastructure for hosting and serving our model recommendation engine." Tomas Hernando Kofman, Not Diamond, Product Hunt, 2024.

The structural tension is cold start versus cost. Scale to zero and you save money but pay a 30-to-90-second cold start on the next request for a large model. Keep a warm replica and the latency vanishes but so do the serverless savings. For raw, cheap GPU time without the managed layer, RunPod is a fraction of the price, but you give up the autoscaling, optimization, and compliance that justify Baseten in the first place.

The friction Baseten teams keep hitting

The billing model is the most common source of surprise. Per-replica-minute costs are hard to forecast, redundancy multiplies them, and there is no prominent budget cap, so teams with spiky traffic report bills jumping without warning. Truss, the same framework that makes onboarding smooth, also creates switching cost, because moving off Baseten means rewriting model wrappers for vLLM or SGLang. Support quality reads as inconsistent in public reviews, with one user describing a ten-day ordeal over a routine change.

"All I needed was a simple billing address update, something every normal platform lets you change directly in the UI." Valerio, Product Hunt, 2025.

Finally, Baseten is engineering-only by design. There is no self-service dashboard for non-technical users, and its monitoring surfaces GPU utilization and response times rather than business metrics. That is correct for its audience and a wall for anyone expecting a no-code experience.

Who should build on Baseten, and who should not

Baseten is a strong choice for ML engineering teams deploying custom or fine-tuned models, for companies whose compliance requirements such as HIPAA, SOC 2, or in-VPC hosting rule out shared inference clouds, for teams building multi-model pipelines with Chains, and for labs running reinforcement post-training that want training-to-inference continuity through Loops. Seed-to-Series-A startups that qualify for the credit program get a generous on-ramp.

It adds little for teams that only call off-the-shelf model APIs, where the native provider is simpler. Price-sensitive teams running standard open models at moderate volume will usually find Together AI or a per-token host cheaper and less operationally involved. Solo developers and hobbyists will find the per-replica costs and engineering overhead excessive, and anyone wanting rapid, code-light prototyping will move faster on Modal. Baseten earns its premium only when you genuinely need managed, compliant, optimized serving of your own models.