Braintrust

Braintrust is a closed-source, end-to-end evaluation and observability platform for teams building LLM-powered products, organized around a blunt premise: you cannot tell whether an AI feature is getting better or worse without systematic measurement, and traditional software tests do not apply to probabilistic outputs. It was founded in San Francisco by Ankur Goyal, whose previous company Impira was acquired by Figma, and it gives engineering teams a structured loop, capture traces, define what good looks like with scorers, run experiments against versioned datasets, and gate releases on quality. The customer roster is unusually strong for a young category, including OpenAI, Stripe, Notion, Vercel, Airtable, and Zapier, which is the clearest signal that the eval-first approach resonates with serious teams.

The platform combines logging and tracing on Brainstore, a proprietary database the company says is markedly faster at querying trace data, with an evals framework, versioned datasets, a prompt playground, and automated scorers spanning LLM-as-judge, code, and human review. CI/CD quality gates can block a deploy when scores regress, Topics automatically surfaces failure clusters from production traffic, and Loop, an in-product AI agent launched in November 2025, analyzes traces in plain language and proposes eval datasets and scorers. SDKs cover Python, TypeScript, Go, Ruby, and C#, the open-source autoevals library works standalone, and the company raised an $80 million Series B at an $800 million valuation in February 2026.

What Braintrust does for AI teams in June 2026

The center of gravity is the eval, not the trace. Where most observability tools start from production logging and bolt on evaluation, Braintrust builds from the experiment workflow outward, which is why it fits teams that treat quality as something to measure and enforce rather than just watch.

"I've never seen a workflow transformation like the one that incorporates evals into mainstream engineering processes before." Malte Ubl, Engineering at Vercel, October 2024.

In practice that means you instrument your app, pull real conversations into a dataset, define scorers, and run experiments that produce side-by-side diffs with statistical significance. A GitHub Actions integration runs those evals on every pull request and posts results as a comment, so a prompt change only ships if it clears a quality bar. Topics, which reached general availability in June 2026, finds failure patterns without manual tagging, and the Loop agent, launched November 24, 2025, turns "have hallucinations increased this week" into an actual analysis with proposed fixes. The $80 million Series B led by ICONIQ in February 2026, with Andreessen Horowitz and Greylock returning, funds the roadmap behind all of it.

Braintrust versus Langfuse and Arize Phoenix

Against Langfuse, the split is philosophy and licensing. Langfuse is tracing-first and fully open-source under MIT, self-hostable by anyone, and was acquired by ClickHouse in January 2026, while Braintrust is eval-first, closed-source on its proprietary Brainstore engine, with no free self-hosted tier. Teams that want open-source freedom start with Langfuse; teams that want the deepest native scoring and experiment tooling start with Braintrust. Against Arize Phoenix, the contrast is discovery versus enforcement. Phoenix is OpenTelemetry-native, free to self-host, and carries ML-drift breadth from Arize's monitoring roots, making it strong at finding what is wrong in production. Braintrust is stronger at systematically fixing it, with more mature CI/CD gates and dataset workflows. A useful shorthand: Phoenix surfaces the problem, Braintrust closes the loop on it.

What the eval-first workflow costs

Braintrust has a genuinely free Starter tier with unlimited users, 1 GB of processed data and 10,000 scores per month, and 14-day retention. The paid jump is steep: Pro is a flat $249 per month, again with unlimited users, raising the included data to 5 GB and scores to 50,000 with 30-day retention, plus RBAC and priority support. Enterprise adds custom retention, hybrid or on-premise deployment, and a premium SLA.

"Evals are the centerpiece of systematic AI Engineering." Ankur Goyal, founder of Braintrust, October 2024.

The honest wrinkle is the usage model. On top of the flat fee, data ingestion and scores bill per unit, and agent workloads that generate many spans and scores per run can pile up overages quickly, with 450,000 scores beyond the Pro allowance adding roughly $675 to the bill. There is also no intermediate tier between free and $249, which makes the step up abrupt for small teams.

Where Braintrust frustrates teams

The most common complaints cluster around cost and support. Pricing is hard to model before signing up because it mixes a flat fee with per-gigabyte and per-score charges, and the leap from free to $249 with nothing in between catches small teams off guard. Support quality draws repeated criticism in third-party reviews, with slow response times described as a systemic pattern rather than isolated incidents, typical of a company that grew to 165 people in roughly a year. The trace UI is optimized for scoring and experiments rather than raw production debugging, so locating a root cause in a 2,000-span agent run takes more navigation than a dedicated tracing tool. There is no self-hosting below the enterprise tier, which is a hard blocker for budget-constrained or data-residency-bound teams, and because evaluation happens after the fact, Braintrust does not provide real-time guardrails that block an unsafe response before it reaches a user.

Who Braintrust is built for, and who should look elsewhere

Braintrust is a strong fit for engineering teams that want evaluations wired into CI/CD, for organizations building an eval-first culture rather than doing ad-hoc prompt testing, for teams comfortable at the $249 per month tier, and for anyone building on any LLM provider rather than being locked into one framework. The framework-agnostic SDKs and GitHub-native gating are the draw.

It is a poor fit for teams that primarily need real-time guardrails or content filtering, since Braintrust evaluates after the fact, and for regulated teams that require self-hosting below an enterprise contract. Organizations running traditional ML alongside LLMs that want unified monitoring are better served by Arize or Weights and Biases. Solo developers and small teams that cannot justify $249 and will quickly exhaust the free tier on agent workloads should look at an open option like Opik or Langfuse instead.