Opik

Opik is an open-source LLM observability and evaluation platform from Comet, the New York company founded in 2017 and known for machine-learning experiment tracking. As teams shifted from training their own models to calling GPT, Claude, and Gemini through APIs, Comet built Opik to bring the same rigor to LLM applications: trace every call, score output quality automatically, and optimize prompts systematically instead of guessing. The license is the cleanest in its category, fully Apache 2.0 with no enterprise code fenced off in the same repository, and the project is genuinely popular, passing 19,700 GitHub stars by mid-2026 and shipping releases almost daily.

Opik covers tracing as span trees with multi-turn thread grouping, processing tens of millions of traces a day at scale, more than 30 LLM-as-judge metrics, online evaluation rules that score a configurable sample of production traffic, and a real-time guardrails layer that blocks PII and off-topic content before it reaches users. Its standout feature is the Agent Optimizer, an SDK with six algorithms that automatically searches for better prompts against a metric you define, paired with a no-code Optimization Studio. It integrates with more than 70 frameworks including LangChain, LlamaIndex, Dify, and Flowise, self-hosts free under Apache 2.0, and offers a Comet Cloud free tier with paid Pro at just $19 per month.

What Opik gives developers in June 2026

The workflow is the familiar trace, evaluate, improve loop, but Opik pushes harder on the improve step than most rivals. Tracing captures every LLM call, tool invocation, and retrieval as a structured span, grouped into conversation threads for multi-turn apps. The 30-plus judge metrics cover hallucination, answer relevance, context recall, moderation, and factual correctness, and online evaluation rules apply them to live traffic on a sampling rate with webhook alerts. Then comes the differentiator: the Agent Optimizer runs meta-prompting, few-shot Bayesian, evolutionary, and other algorithms to automatically test prompt variants against your metric, with a no-code Optimization Studio added in December 2025 and a Cost Intelligence feature in June 2026 that tracks Claude Code and Codex spend by team. Opik launched publicly on September 17, 2024, posted to Hacker News three days later, and went from zero to 12,500 stars in roughly nine months, one of the fastest-growing projects in the space.

Opik versus Langfuse and Braintrust

Against Langfuse, the licensing is the cleaner story for Opik: Opik is Apache 2.0 across the whole repository, while several Langfuse enterprise modules live under a commercial license in dedicated folders, so for strict open-source or air-gapped use Opik is legally simpler. Opik also ships the automated Agent Optimizer, which Langfuse has no equivalent for, though Langfuse counters with a stronger prompt-management UI and a ClickHouse backend that scales analytics further. Against Braintrust, the gap is cost and openness. Braintrust is proprietary with no free self-hosted tier and a $249 per month entry, where Opik is fully open and $19 per month on cloud. Braintrust still leads on CI/CD quality gates and side-by-side dataset comparison, while Opik answers with automated optimization, full online evaluation, and broad multimodal support.

The price and license advantage

Opik's pricing is its most disarming feature. The open-source build is free and Apache 2.0, self-hosted via Docker Compose for local work or a Helm chart for production. Comet Cloud is free up to 25,000 spans per month for ten users with 60-day retention, and Pro is just $19 per month for 100,000 spans and fifty users, with extra spans at $5 per 100,000. Enterprise adds SOC 2, ISO 27001, HIPAA, SSO, and dedicated support.

"Setting up Opik was surprisingly straightforward. When I implemented automated evaluation on my research assistant project, I discovered that about 30% of my retrieved documents had questionable relevance, which led me to retune my embedding model." Md Zahid Hussain, Medium, July 2025.

That $19 entry sits far below Braintrust's $249 and LangSmith's per-seat pricing, and academics get the full Pro plan free. For a team that wants real evaluation without a procurement conversation, the cost barrier essentially disappears.

Where Opik still has gaps

The most cited limitation is authentication in the self-hosted build, which lacks built-in multi-user management.

"I would like to enable on my self hosted: workflows, authentication, API key. I tried to find it in the documentation but I couldn't find it." GitHub issue 976, January 2025.

Teams that want a shared self-hosted instance must add an external auth layer, and the default Docker Compose deployment runs unauthenticated, which is fine locally but a risk if exposed. Automatic cost tracking has been limited to OpenAI and Google models, so teams on Anthropic, Mistral, or custom deployments need manual mapping, and configuring custom evaluation metrics carries a learning curve. With 490 releases and no documented long-term-support channel, self-hosted deployments that fall behind on server updates can hit subtle SDK-to-server version mismatches. Dashboard customization, only recently added in beta, is still maturing.

Who should reach for Opik

Opik is an excellent fit for Python-first teams building RAG pipelines, chatbots, or agent workflows who want a fully open-source, Apache-2.0 tool with no vendor lock-in, for budget-conscious teams that balk at $249 per month elsewhere, for anyone who wants automated prompt and agent optimization out of the box, and for existing Comet experiment-tracking users who get a unified platform. Data-residency-bound teams can self-host the whole thing on Kubernetes.

It is a weaker choice for JavaScript-only stacks, since the Python SDK is the mature one, for teams that need multi-user authentication in self-hosting today without a do-it-yourself layer, and for shops running mostly non-OpenAI or non-Google models that need automatic cost tracking. Non-technical builders who want a no-code app builder should look elsewhere, and teams whose main need is traditional ML drift monitoring are better served by Arize Phoenix or Weights and Biases.