The five agent gateways that make multi-agent systems governable in 2026
Multi-agent systems move a growing share of traffic from users through other agents. An agent gateway governs that traffic the same way an API gateway governs service-to-service calls, and it has to understand A2A, MCP and the LLM traffic the agents produce.

TL;DR
- A multi-agent system produces three traffic types through the same control point: model calls to LLM providers, tool calls over MCP, and inter-agent calls over A2A or a vendor equivalent. An agent gateway governs all three.
- The useful test is whether one key, held by the calling agent or its user, carries model budgets, allowed tools and allowed subagents together. Where those three are split across different systems, nothing stops an under-budget agent from delegating to an over-budget one.
- Bifrost ranks first because the same process routes LLM, MCP and agent traffic, one virtual key carries models, budgets and tool permissions, agent mode runs tool loops with explicit allow-lists, and code mode cut a 508-tool run to $29 in Maxim AI’s published benchmark.
- Cloudflare’s Agents platform and Kong’s AI Gateway cover the networking and policy side well, but delegate LLM and tool permissions to separate products. Amazon Bedrock AgentCore and LiteLLM round out the field for teams committed to those stacks.
Why multi-agent systems need a gateway
Multi-agent systems route work from a planner to specialist agents, and each of those agents makes its own model calls, tool calls and, often, further delegations. An agent gateway is the point where those three traffic types are governed together.
The pattern is familiar from service-to-service networking. An API gateway in front of microservices terminates mTLS, applies quotas, logs each call and keeps credentials out of the client. An agent gateway does the same thing for the agent runtime, except the clients are LLM applications, the servers are other agents or MCP tool servers, and the protocols are different. The Agent-to-Agent protocol, now at its 0.2.9 revision according to the A2A specification, defines capability discovery, streaming task updates and authentication between agents. The overview page describes A2A as “an open protocol enabling communication and interoperability between opaque agentic applications”, which is the point: a caller does not have to know how a subagent is implemented, only how to talk to it.
MCP is the other half. The current Model Context Protocol specification covers the client-to-server path to tools, with tools described as “arbitrary code execution” and authorization explicitly optional. Any gateway that aggregates tools and subagents has to turn that optionality into a policy a human has approved.
The useful mental model is one request, three protocols. A user asks a planner agent to summarise a sales pipeline; the planner calls an analytics agent over A2A, which calls a database tool over MCP, which produces results summarised by a model call over HTTPS. If budgets and tool permissions live in different places, a key limited to a cheap model can still drive an agent that calls a database tool with no budget attached. The cleanest fix is a gateway that holds all three in one record.
Agent runtimes are not standardised yet, and each vendor’s view is coloured by the agent framework it ships. The companion piece on agent orchestration frameworks covers the upstream half of the choice. This comparison focuses on the gateway in the middle: what it terminates, which identities it recognises and what it does with tool and subagent calls.
How the comparison was scored
The five products were scored against criteria written before the list was filled, so readers can weight them differently.
| Criterion | What was checked | Why it matters |
|---|---|---|
| A2A support | Whether the gateway speaks A2A or an inter-agent equivalent at the protocol level | Multi-agent systems need agent-to-agent, not just agent-to-tool |
| MCP tool governance | Per-caller tool allow-lists, token exchange, deny-by-default | Tools represent code execution and permissions move with the caller |
| Unified key for LLM, MCP and subagents | One record of model budget and allowed tools and subagents | Prevents under-budget agents from delegating to over-budget ones |
| Inter-agent authentication | Delegation of user identity, token exchange between agents | Avoids shared service accounts between agents |
| Logging and audit | Traced requests across agents, signed audit of policy changes | Required for incident response and compliance |
| Deployment model | Self-hosted, hosted, managed | Decides where agent prompts and results travel |
Published benchmark data, where it exists, is cited from the vendor that ran it. No product in this comparison was load-tested by this publication. Running the same gateway choice in front of every model call, not only agent traffic, is the pattern described in the companion piece on running an AI gateway in front of every model call.
Five agent gateways at a glance
| Rank and gateway | Deployment and license | A2A or inter-agent | MCP governance | LLM gateway in same product |
|---|---|---|---|---|
| 1. Bifrost | Self-hosted binary, Docker or Go SDK; Apache 2.0 core, paid enterprise | Agent mode, MCP-based delegation, token exchange | Per virtual key, deny by default, Virtual MCPs | Yes |
| 2. LiteLLM | Self-hosted Python proxy; MIT core, enterprise license | MCP permissions by key, team, organization | Per key or team allow-lists; token exchange | Yes |
| 3. Cloudflare AI Gateway | Managed, Cloudflare One | MCP server portals and Agents platform on Workers | Portal-level tool allow-lists | Yes |
| 4. Kong AI Gateway | Kong Gateway 3.12+; AI features in Enterprise | AI MCP Proxy and OAuth2 plugins | Per-tool ACLs on consumers | Yes |
| 5. Amazon Bedrock AgentCore Gateway | Fully managed AWS service | A2A interoperability, HTTP targets for agents | Cedar policies and interceptors on tools | Yes |
1. Bifrost
Bifrost, the open-source gateway written in Go by Maxim AI, is the strongest general-purpose agent gateway in the comparison because one process handles model, tool and subagent traffic under one set of keys. The overview documentation describes it as “one interface to 10,000+ models across 25+ providers” with an MCP gateway in the same binary. The core is Apache 2.0; clustering, SSO, signed audit logs and guardrails belong to a paid enterprise tier.
One key for models, tools and subagents
Bifrost’s virtual keys bundle the three things a multi-agent system has to govern together: which models and providers a caller may use, which budget applies and which MCP tools and subagents the caller may call. Budgets and limits can be set at the virtual key, team or customer level, and are checked on the request path, so a planner agent that exceeds its spend does not get another model call from a subagent on the same key.
The deny-by-default rule in MCP tool filtering carries through to agent delegation. A virtual key with no MCP configuration sees no MCP tools, except those an administrator has marked “Allow by Default.” A request header can narrow the allowed set but never widen it, and the allow-list is checked again at execution time. The combined key model is what Bifrost’s governance page calls “identity-based governance running off the identity provider already in use”, meaning the same identity carries everything an agent is allowed to do.
Agent mode and code mode
Bifrost’s agent features sit in the MCP layer. Agent mode lets the gateway drive the tool loop itself, but only for tools listed in tools_to_auto_execute, which must be a subset of the tools the caller may execute. Everything else is returned for human approval, and the loop stops at a configurable maximum depth of ten by default. Streaming responses are disabled in agent mode, which the documentation flags explicitly.
Code mode is the token-cost half of the story. Instead of sending every tool definition to the model, Bifrost exposes four meta-tools; the model reads tool stubs on demand and writes a short Python (Starlark) script that Bifrost runs in a sandbox. Maxim AI’s published benchmark reports a 58.2 percent fall in input tokens with 96 tools across six servers and a 92.8 percent fall with 508 tools across 16 servers, a 100 percent pass rate in both and a cost reduction from $377 to $29 on the 508-tool run. On a long-running agent, the cost shift matters more than the latency shift.
Performance and governance
The t3.xlarge benchmark page reports 11 microseconds of added overhead per request at 5,000 requests per second on an AWS t3.xlarge, with 100 percent success. The same comparison puts Bifrost’s throughput at 424 requests per second against 44.84 requests per second for a Python-based alternative, with memory at 120 MB against 372 MB. Observability emits OpenTelemetry GenAI spans and Prometheus metrics, and retries and fallbacks run with weighted load balancing across providers, so a provider outage does not take down one agent’s path through the system.
Enterprise tier: clustering, SSO, group-level access profiles, token exchange and signed audit logs (with SIEM export and SOC 2 Type II, ISO 27001, HIPAA and GDPR compliance) are part of Bifrost Enterprise. Evidence qualifications: the performance numbers are from Bifrost’s own published benchmark on a t3.xlarge; neither this publication nor any reader should treat them as general across every workload.
Best for: teams running multi-agent systems in production that want tool, model and subagent permissions on the same key, open-source code they can self-host, and an OAuth 2.1 flow that lines up with the MCP authorization model.
2. LiteLLM
LiteLLM is a Python LLM proxy with an MCP gateway in the same product. Its MCP documentation supports Streamable HTTP, SSE and stdio, with permissions managed by key, team or organization; upstream authentication covers OAuth with PKCE, client credentials, API keys, AWS SigV4 and RFC 8693 token exchange, and MCP usage feeds its cost tracking. Virtual keys govern model access, spend and rate limits, and can be filtered by MCP server to restrict which tools each key may call.
LiteLLM does not speak A2A natively; its agent story runs through the model layer, where a planner agent can call subagent endpoints as if they were models. That is workable for small deployments and matches how teams on earlier agent frameworks already run. The trade-off is that inter-agent authentication and A2A’s streaming-task semantics are not enforced at the gateway, so an operator has to write that logic in the agent code.
The proxy itself has operational constraints familiar to anyone who has read the companion piece on LiteLLM alternatives: Python workers, Redis for shared rate-limit state and Postgres for spend and config. For a team already standardised on it, the MCP gateway adds useful per-key controls without a new product; for a team starting from scratch, a Go gateway is a lighter runtime.
Best for: teams already on LiteLLM that want MCP permissions under the same keys and are willing to treat subagents as model endpoints.
3. Cloudflare AI Gateway and Agents
Cloudflare AI Gateway is a managed proxy for model traffic that works alongside Agents on Workers, Cloudflare’s agent runtime. The combination is the most credible hosted option for teams that already run on Cloudflare. AI Gateway handles 50+ providers with BYOK and emits standard logs for every request; the Agents platform adds durable objects for per-agent state, MCP server portals for tool access and the Zero Trust identity model for callers.
The split across products is also the trade-off. AI Gateway governs model traffic, MCP server portals govern tool access, and the Agents platform is where the agent code runs. Subagent-to-subagent calls happen inside Workers, which means they are governed by Cloudflare’s service tokens and Zero Trust policies rather than a single agent key. For teams that treat Cloudflare as their platform, that is a feature, since existing identity policies carry straight over; for teams that want one key for everything, it is more pieces to wire together.
Cloudflare’s approach to tool bloat also deserves a mention. Portals support a code-mode option that replaces tool definitions with JavaScript execution, which is on by default in the current documentation. The effect is similar to Bifrost’s code mode on tokens, scoped to Cloudflare’s runtime.
Best for: teams already on Cloudflare One that want agent, model and tool governance in one platform, provided they are comfortable with a managed service in the data path.
4. Kong AI Gateway
Kong AI Gateway is Kong Gateway with a set of AI plugins that sit in front of model and tool traffic. Its MCP support arrived in Gateway 3.12 through two plugins, both limited to its AI Gateway Enterprise tier. The AI MCP Proxy plugin can pass MCP traffic through to existing servers, convert REST API paths into MCP tools or aggregate converted tools into one MCP server. From 3.13 it supports per-tool ACLs that allow or deny consumers and consumer groups, with every access attempt written to an audit log; from 3.14 authorization failures return HTTP 403. The companion AI MCP OAuth2 plugin makes Kong the OAuth resource server for MCP traffic, rejects invalid tokens with a 401 and does not forward access tokens upstream by default. Since 3.14 it can exchange the client’s token for a different one before forwarding.
For agent-to-agent traffic specifically, Kong does not implement A2A. Its model is that agents are services; one agent calls another through an endpoint that Kong already fronts, and the AI plugins apply per-consumer policies to the HTTP path. That is a reasonable starting point if the agents in question are already HTTP services and Kong is the team’s gateway. It is not a drop-in A2A implementation, and the operational model stays in Kong’s consumer and consumer-group abstraction.
Best for: organisations whose APIs and service traffic already run through Kong and that want to add governed model and tool access to the same gateway, with Enterprise licensing in scope.
5. Amazon Bedrock AgentCore Gateway
Amazon Bedrock AgentCore is AWS’s agent runtime and control plane for agents in production, and AgentCore Gateway is the component that governs tool and inter-agent traffic. The gateway turns OpenAPI, Smithy and Lambda targets into MCP tools, aggregates existing MCP servers into one virtual server, fronts other agents through HTTP targets and routes inference requests across model providers. The AgentCore interoperability page lists A2A as a supported inter-agent protocol alongside MCP, with the caveat that the pieces are tied to AWS accounts and IAM.
The strengths are the ones a managed cloud service usually offers: fine-grained access control works at gateway, tool, operation and parameter level through request interceptors that read JWT claims, or through Cedar policies on IAM principals, and AWS recommends that interceptors deny by default. Observability and audit are integrated with CloudWatch and CloudTrail. Semantic tool search means agents can find tools without loading every definition.
The constraints are the obvious ones. AgentCore runs on AWS, agent prompts and results travel through AWS services, and tool-level access is something a team builds in interceptor logic rather than a setting on a role. For teams that have committed to Bedrock as the model layer, the lack of a proxy to operate is the point; for teams that want to self-host the control plane, it is a non-starter.
Best for: teams building agents on AWS that want a managed control plane tied to IAM, CloudWatch and CloudTrail.
Choosing an agent gateway by starting point
| Starting point | First agent gateway to evaluate | Reason |
|---|---|---|
| Multi-agent system calling models, tools and other agents | Bifrost | One key for models, tools and subagents; open-source self-hosting |
| Existing LiteLLM proxy | LiteLLM | MCP permissions on the same keys |
| Cloudflare-native agents and workers | Cloudflare AI Gateway and Agents | Identity and portals already in place |
| Existing Kong estate | Kong AI Gateway | Per-tool ACLs on existing consumers |
| AWS-native, managed only | Amazon Bedrock AgentCore | A2A support, managed control plane |
A regulated team has four questions to ask each vendor. Does one key record model budgets, tool permissions and subagent permissions, or do they live in separate systems. Does the gateway refuse to pass a caller’s token to an upstream agent or tool unchanged. Does it deny unknown tools and unknown subagents by default. Can it produce a signed trail of who called what through which agent. The fifth, worth adding for a long-lived deployment, is how the gateway moves when the A2A or MCP specifications change: Microsoft’s MCP gateway already refuses older clients, and A2A’s 0.2.9 revision will not be the last.
Verdict
Bifrost is the strongest agent gateway in this comparison because it is the only open-source option that pairs a standards-based OAuth 2.1 flow for MCP clients, delegated and per-user credentials for upstream servers, deny-by-default tool access per virtual key and a full LLM gateway in the same process. Agent mode makes the tool loop explicit, code mode addresses the token cost of large catalogs, and signed audit logs in the enterprise tier close the compliance gap.
The alternatives are credible within their niches. LiteLLM suits teams already on it that want MCP permissions on the same keys. Cloudflare’s combination of AI Gateway, Agents on Workers and MCP portals is the most complete managed offering for teams on that platform. Kong extends an existing API gateway to agent traffic under an Enterprise licence. Amazon Bedrock AgentCore is the managed option for teams fully committed to AWS.
Limits of this comparison
Capabilities come from vendor documentation read in November 2026. No gateway was deployed, load-tested or penetration-tested, and no vendor benchmark was reproduced. Bifrost’s throughput and overhead figures are from Maxim AI’s own published t3.xlarge benchmark and should not be generalised to every workload. A2A’s specification is at version 0.2.9 and will continue to move; MCP has already changed enough between 2025 and the 2026-07-28 revision that gateways are divided on which clients they accept. Pricing was not compared.
Sources
- A2A protocol specification
- Agent2Agent (A2A) protocol overview
- Model Context Protocol specification (2026-07-28)
- Bifrost source repository (GitHub, Apache 2.0)
- Bifrost docs: overview
- Bifrost docs: MCP overview and gateway mode
- Bifrost docs: MCP agent mode
- Bifrost docs: MCP code mode and benchmark
- Bifrost docs: virtual keys
- Bifrost docs: budgets and limits
- Bifrost docs: retries and fallbacks
- Bifrost docs: built-in observability
- Bifrost docs: t3.xlarge benchmark
- Maxim AI: Bifrost AI gateway
- Maxim AI: Bifrost MCP gateway
- Maxim AI: Bifrost governance
- LiteLLM: proxy server overview
- LiteLLM: MCP gateway
- LiteLLM: virtual keys
- Cloudflare AI Gateway documentation
- Cloudflare: Agents on Workers
- Cloudflare: AI Gateway BYOK and providers
- Kong: AI Gateway overview
- Kong: AI MCP Proxy plugin
- Kong: AI MCP OAuth2 plugin
- AWS: Bedrock AgentCore overview
- AWS: AgentCore Gateway
- AWS: AgentCore interoperability
Questions readers ask
What is an agent gateway?
An agent gateway is a proxy between agents and the services they talk to, including other agents. It terminates inter-agent protocols such as A2A, authenticates the calling agent, applies per-caller permissions to the tools and subagents it exposes, manages credentials for upstream services, and records every call for audit and billing. It usually sits in the same process as an LLM gateway and an MCP gateway, because a multi-agent request produces model calls and tool calls on the way to the final answer.
How is A2A different from MCP?
MCP (Model Context Protocol) is a client-to-server protocol between an LLM host and a tool server. A2A (Agent-to-Agent) is a peer protocol between agents that exposes capabilities, streams work and handles authentication between agents. A gateway that handles both lets one caller invoke a tool directly and delegate a longer task to a subagent under the same keys and budgets.
Why not route A2A through an existing API gateway?
API gateways route HTTP endpoints but do not understand A2A capability discovery, streaming task updates, inter-agent authentication or the LLM traffic that agents produce as a side effect. A gateway that understands both keeps model, tool and subagent permissions on the same key, so a cheap agent cannot delegate to an expensive one without that budget showing up.
Do I need an agent gateway for a single agent?
A single agent calling a few tools does not need one. The case appears when agents call other agents, when many teams share agents, when inter-agent authentication has to be delegated per user instead of using a shared service account, and when security teams need one place to record which agent did what.
Which agent gateways are open source?
Bifrost is Apache 2.0, LiteLLM has an MIT core outside its enterprise directory, and Kong Gateway is open source with MCP and AI features limited to its Enterprise tier. Cloudflare AI Gateway and Amazon Bedrock AgentCore are managed services. All five can be run in front of agent traffic in production.
