Helicone

Helicone is an open-source LLM observability platform and AI gateway built by Justin Torre and Cole Gottdank, launched through Y Combinator's W23 batch in 2023. The core problem it solves is a practical one: when teams start shipping AI applications, they quickly lose visibility into what their LLMs are actually doing, what it costs, and where it breaks. Helicone restores that visibility through a proxy-based architecture, routing requests through its Cloudflare Workers-based gateway so every call to OpenAI, Anthropic, Gemini, Mistral, or DeepSeek gets logged, timed, and analyzed automatically. Before the company was acquired by Mintlify on March 3, 2026, Helicone had processed 14.2 trillion tokens across 16,000 active organizations.

The platform's headline feature is a one-line integration: swap your OpenAI base URL to point at Helicone's gateway and your entire application is instrumented instantly, no SDK wrapping required. Beyond basic logging, Helicone provides semantic caching (which teams regularly report cutting LLM costs 20-40%), AI Gateway routing with automatic provider failover, prompt versioning and management, HQL (Helicone Query Language) for custom analytics queries, rate limiting, and a playground for testing prompt variants against real production data samples. The platform is Apache 2.0 licensed and fully self-hostable via Docker or Helm. Since its March 2026 acquisition by Mintlify, Helicone operates in maintenance mode with continued security patches, new model support, and bug fixes, but no major new feature development.

What Helicone actually does in April 2026

Helicone functions as a transparent HTTP proxy sitting between your application and any LLM provider. When you call openai.baseURL = "https://oai.helicone.ai/v1", the request passes through Helicone's Cloudflare Workers edge network, gets logged to ClickHouse for analytics, and is forwarded to OpenAI with sub-5ms overhead at the p50 latency benchmark. The platform supports over 100 models through its unified AI Gateway, including OpenAI, Anthropic, Azure OpenAI, Google Gemini, DeepSeek, Mistral, Together AI, OpenRouter, and Groq. Integration support extends to 30+ frameworks including LangChain, LlamaIndex, the Vercel AI SDK, and LiteLLM.

The observability dashboard surfaces cost, latency, and quality metrics broken down by user, session, endpoint, and model. Teams use HQL to write custom queries against their request history, similar to SQL analytics on top of their LLM logs. The Prompts feature allows prompt versioning tied to real production data, with deployment of new prompt versions directly through the gateway without code deploys. Helicone's Datasets feature lets teams build evaluation sets from sampled production traffic, while the Playground supports live testing of prompt variants before promoting them. Built-in alerts and reports cover cost spikes, error rates, and latency anomalies.

The platform earned its place in the ecosystem partly through timing: it launched when logging LLM calls required building custom middleware, and it genuinely solved that problem with minimal friction. By the time of its acquisition, it was the most-used LLM observability tool among YC portfolio companies and had been #1 on Product Hunt on launch day.

"Helicone just works right out of the box, really helpful for us to dig into user issues" - Brandon Chen, Product Hunt, 2024

Where Helicone sits versus Langfuse and LangSmith

The mechanical difference between these three platforms is architectural, not cosmetic. Helicone intercepts at the network layer, sitting between your application code and the LLM provider's API endpoint. It sees individual HTTP requests and responses. Langfuse and LangSmith integrate at the application layer, wrapping agent steps and workflow logic inside your code with SDK calls. This distinction determines what each platform can actually observe.

Langfuse (MIT license, 19K+ GitHub stars, self-hostable) requires explicit SDK instrumentation: developers wrap each agent step, trace span, and evaluation hook in Langfuse SDK calls. This takes 1-2 hours to set up but gives complete multi-step agent workflow visibility with nested span hierarchies. Langfuse sees that LLM call B was triggered by tool call A inside agent run X, which Helicone's proxy cannot capture without manual header tagging. Langfuse also has a more mature prompt management and dataset evaluation system. The self-hosted option is fully free; cloud starts at $50/month with a generous free tier of 50K events/month. For teams who want to self-host their observability data in their own infrastructure, Langfuse is the clear choice.

LangSmith (LangChain's observability product, $39/month Plus) is framework-coupled. For teams already using LangChain or LangGraph, tracing is near-zero-config via two environment variables. LangSmith understands LangChain chain topology natively, knows which tool triggered which LLM call, and surfaces that context automatically. For teams not using LangChain, manual instrumentation takes 2-3 hours and LangSmith loses its main advantage. In March 2025, LangChain added end-to-end OpenTelemetry support to LangSmith, reducing vendor lock-in somewhat. LangSmith's built-in evaluation harness and per-run playground are deeper than Helicone's equivalent features.

Helicone's advantages over both are setup speed (under 15 minutes, zero application code changes) and built-in semantic caching, which neither Langfuse nor LangSmith offer natively. At $79/month for unlimited seats, Helicone is also cheaper than both for growing teams. The tradeoff: Helicone sees individual API calls, not workflow topology. If your application is a simple chain of LLM calls, Helicone gives you everything you need. If it is a complex multi-step agent where you need to understand why step 7 triggered a particular model call, you need Langfuse or LangSmith.

"It gives me extremely useful and detailed insights on usage, costs, and response times" - Ryan Hendrickson, Product Hunt, 2024

What the proxy reality looks like day to day

The integration experience genuinely delivers on its promise. For a team running OpenAI calls, the change is one line in the OpenAI client constructor. For an Anthropic team, similarly one header swap. From that moment, every request appears in the Helicone dashboard. Cost tracking is automatic: Helicone maps every model and token count to its current provider pricing and shows you exactly what each endpoint, user, or feature costs per day. Latency percentiles (p50, p90, p99) are broken out by model and request type.

Semantic caching is the feature most teams cite when describing ROI. Helicone embeds each request, checks the cache for semantically similar prior responses within a configurable similarity threshold, and returns the cached response if the threshold is met. Teams with high-repetition workloads (customer support bots, document Q&A systems, coding assistants) commonly report 20-40% cost reductions within a week of enabling it. Configuration is through the dashboard, not code.

The frustrations that surface in production tend to cluster around what the proxy architecture cannot see. When an agentic workflow fails after seven steps, the Helicone dashboard shows seven individual API calls with their costs and latencies, but not how they are connected or which upstream decision triggered the failure. Debugging multi-step agent failures through Helicone requires manual session tagging and trace correlation, which erases the "no code changes" advantage. Teams who reach this complexity typically add Langfuse as a second layer or migrate entirely.

Who Helicone is built for

Helicone is the right tool for teams who need LLM observability quickly and do not want to instrument their codebase. The primary beneficiaries are early-stage AI product teams (2-20 engineers), teams running simple LLM pipelines rather than complex multi-step agents, and cost-conscious teams who want caching and analytics without building infrastructure.

It works especially well for multi-model shops running requests across several providers. The unified dashboard aggregates OpenAI, Anthropic, and other provider costs into a single view, which is harder to replicate with provider-native tools alone. For startups that are iterating quickly on prompts, the Playground and Datasets features let non-engineers run prompt experiments against production data samples without touching code. For teams under 10 people who just need to answer "what is our LLM spend this month and which feature is causing it," Helicone is arguably overbuilt but completely adequate.

SOC-2 and HIPAA compliance are on the Team plan at $799/month, which prices out smaller teams who need compliance. Enterprise organizations with strict data residency requirements in the EU may find the proxy model uncomfortable even with Cloudflare's edge network: every LLM request, including user content, passes through Helicone's infrastructure. The HN launch thread in March 2023 surfaced these concerns explicitly, with multiple commenters asking about EU PII and whether the proxy architecture violated OpenAI's terms of service. Helicone addressed both questions, but the architectural reality of a cloud proxy handling production traffic remains a consideration for data-sensitive use cases.

What Helicone is not

Helicone is a monitoring layer, not a full-stack observability platform. It does not support prompt evaluation pipelines with automated scoring, regression detection, or evaluation suite quality measurement. Its Scores feature allows attaching manual numeric ratings to requests, which is basic compared to Langfuse's or Braintrust's evaluation harnesses. When a request fails or produces a bad output, Helicone logs it, but there is no issue state machine, no lifecycle tracking, no mechanism to determine whether a recurring problem was resolved or regressed.

The maintenance-mode status since March 2026 is also relevant context. Teams choosing Helicone today are choosing a product in wind-down: security patches and new model additions will continue, but the product roadmap stopped. For teams with a multi-year horizon who need active development of the observability platform they depend on, this is a real constraint. The Helicone team explicitly named LiteLLM and Portkey as alternatives for teams seeking continued active development.

Getting started with Helicone

The onboarding path is the platform's strongest feature. Create a free account at helicone.ai, copy your Helicone API key, and swap your OpenAI client's base URL to https://oai.helicone.ai/v1 with the Helicone key added as a header. That single change sends all OpenAI traffic through Helicone. For Anthropic, point to https://anthropic.helicone.ai. For Azure OpenAI, Google Gemini, and others, Helicone provides provider-specific gateway URLs. The typical time to first logged request is under five minutes.

The free Hobby tier handles 10,000 requests/month with 7-day retention, enough for development and light production use. The Pro plan at $79/month unlocks unlimited seats, 1-month retention, alerting, and HQL analytics queries. Self-hosting is documented for Docker and Helm with the full five-service stack (NextJS web, Cloudflare Worker, Express API, Supabase, ClickHouse, MinIO). For teams that need to keep data off third-party infrastructure, self-hosting is viable, though it requires maintaining the stack. The GitHub repository at github.com/Helicone/helicone has 5.6K stars and a Rust-based AI gateway component with benchmark results showing under 5ms p50 overhead at high throughput.