Kong AI Gateway

Kong AI Gateway is the LLM and agent traffic-governance layer built on top of Kong Gateway, the widely deployed open-source API gateway from Kong Inc., a San Francisco company founded in 2017. It targets an enterprise problem that grows with every model you adopt: as an organization wires OpenAI, Anthropic, Bedrock, Gemini, Mistral, and others into internal systems and agent workflows, control fragments, because each provider brings its own SDK, credentials, cost meter, and rate-limit behavior. Kong sits in front of all of them as a single control plane for routing, cost containment, security, and observability. It is not a separate binary but Kong Gateway plus a suite of AI plugins, run either self-hosted or through Kong's managed Konnect control plane.

The capabilities span multi-provider routing and load balancing, token-based rate limiting and quotas, semantic caching that skips duplicate calls, PII sanitization and prompt-injection guards, and dashboards for per-model token spend and latency. By April 2026, version 3.14's Agent Gateway extended that governance to two traffic types beyond raw LLM calls: Model Context Protocol tool traffic and agent-to-agent communication, which Kong claims makes it the only gateway covering all three from one control plane. Six core AI plugins are free and open-source on Kong Gateway, while the serious features live in paid Konnect tiers. Kong itself is well capitalized, with a $175 million Series E in late 2024 at a $2 billion valuation.

What Kong AI Gateway governs in June 2026

The mental model is a single chokepoint for all AI traffic. Instead of each service holding its own provider keys, calls flow through Kong, which applies routing, authentication, rate limits, caching, and security policy in one place, then emits unified observability. That is genuinely valuable at scale: you can cap token spend per team, fail over from one provider to another, cache semantically similar prompts to cut cost, and strip PII before it ever reaches a model.

"By simply upgrading your current Kong Gateway to version 3.6 you'll be able to use these new plugins entirely focused on AI and LLM usage." Marco Palladino, CTO of Kong, February 2024.

The most recent and distinctive step came on April 14, 2026, when Kong shipped the Agent Gateway in version 3.14, adding governance of agent-to-agent communication over Google's A2A protocol on top of existing LLM and MCP support. That release brought agent identity and authentication, real-time inspection of agent-to-agent streams for policy violations and prompt injection, per-agent cost allocation, and full audit logging. Kong's September 2025 acquisition of the metering company OpenMeter signals it is also working on the token-cost-tracking side of the story.

The open-source plugins versus paid Konnect

The split between free and paid matters more here than with most tools. Kong Gateway is genuinely free and Apache 2.0, and since version 3.6 in February 2024 it ships six AI plugins at no cost, covering LLM proxying, prompt guarding, and request and response transformation. What the open-source tier does not include is the part most AI teams actually want: token-based rate limiting (the free tier only counts HTTP requests), semantic caching, PII sanitization at scale, the management GUI, SSO, advanced analytics, and the MCP and A2A governance. Those require Konnect, which offers a 30-day free trial, then a self-serve Plus tier that includes one million API requests per month with $200 per additional million and up to five LLM models, and an Enterprise tier with unlimited models, SSO, audit logs, and custom SLAs that typically lands in the tens of thousands of dollars per year. Kong does not charge per token; you still pay providers directly for that.

Kong versus LiteLLM and Portkey

Against LiteLLM, the contrast is weight and language. LiteLLM runs as a lightweight Python process on a small server with PostgreSQL, is Python-native so AI engineers extend it with the tools they already know, and covers a broad provider list. Kong self-hosted means running Nginx, a database, and data-plane nodes, and custom plugins are written in Lua, a niche language in AI circles. Kong's own benchmark shows it far ahead on raw throughput, but that advantage is largely irrelevant when the LLM provider's own latency of hundreds of milliseconds dwarfs a few milliseconds of gateway overhead. Where Kong pulls ahead is mature enterprise governance: RBAC, SSO, audit logs, and the MCP and A2A coverage LiteLLM lacks. Portkey is the other rival, purpose-built for LLM traffic with Python-native plugins and per-request pricing, where Kong is an API gateway adapted to AI with per-service pricing. The honest summary is that Kong wins when you want to unify AI governance with the broader API and microservices governance you already run, and a multi-LLM router like OpenRouter or a backend like Together AI covers a narrower slice of the same need.

Where Kong's model frustrates AI teams

The recurring complaint is that the pricing model was built for traditional APIs, not agents.

"A single Agent workflow might trigger a chain of 20+ internal API calls. This multiplier effect makes Kong economically inefficient for agentic AI applications." TrueFoundry technical analysis, 2026.

Because each model or provider counts as a separate gateway service, and agentic workflows fan out into many internal calls per user action, costs can climb fast, with one analysis estimating roughly $2,625 per month in service fees alone for five LLM integrations across twenty microservices. The Lua requirement for custom plugins is a real onboarding barrier for Python-first AI teams, self-hosting at production scale carries disproportionate operational overhead for teams whose only need is an AI gateway, documentation for AI-specific features lags the mature core, and several capabilities teams expected at launch, like model load balancing and MCP proxying, arrived months later. The pattern of trailing AI-native tools on AI-specific features has narrowed by 2026 but persists.

Who Kong AI Gateway is for, and who it overshoots

Kong is a strong fit for teams already running Kong Gateway for non-AI API management, where adding AI plugins is low marginal cost and the unified observability is a real gain. It suits enterprises that need MCP and agent-to-agent governance, regulated industries that require PII sanitization, audit logs, and RBAC over LLM traffic, and large engineering teams with the platform expertise to write Lua plugins and run the infrastructure.

It overshoots most small AI product teams and startups not already on Kong, where the operational overhead and per-service pricing favor consolidation plays rather than greenfield AI infrastructure, and a simpler tool like LiteLLM fits better. Python-native teams who want to extend the gateway with LangChain-style logic, teams whose real bottleneck is provider latency rather than gateway overhead, anyone expecting per-token pricing at the gateway, and solo developers who would need paid Konnect for the serious features should all look elsewhere. Teams routing to hosted models such as Fireworks AI without enterprise governance needs will find Kong heavier than the job requires.