Independent reporting on artificial intelligence.


The Frontier Wire

Comparisons

OpenRouter, LiteLLM and Bifrost compared: three gateways, three workloads

Three LLM gateways dominate the conversation in late 2026. One is a hosted aggregator, one is a Python proxy, and one is a Go gateway. The choice is less about features than about which workload each is built for.

The Frontier Wire cover for a 2026 three-way comparison of OpenRouter, LiteLLM and Bifrost as production AI gateways.
The Frontier Wire cover for a 2026 three-way comparison of OpenRouter, LiteLLM and Bifrost as production AI gateways.

TL;DR

  • OpenRouter, LiteLLM and Bifrost cover three different starting points: a hosted aggregator that takes the operational work off the team, a Python SDK and proxy for teams that live in Python, and a Go gateway for teams that want control, governance and MCP in one process.
  • The dividing line is where the gateway runs. OpenRouter sits outside a team’s perimeter; LiteLLM and Bifrost run inside it. That decision carries the biggest downstream consequences for governance, residency and cost.
  • Bifrost ranks first for production self-hosted deployments because one process governs models, MCP tools and agent traffic, its published t3.xlarge benchmark reports 11 microseconds of overhead per request at 5,000 RPS, and 424 RPS versus 44.84 RPS for a Python-based alternative in the same comparison.
  • Many teams end up with more than one: Bifrost as the control plane, OpenRouter as one of its providers, LiteLLM kept only where its SDK is embedded in an existing application.

Three gateways, three workloads

Choosing between OpenRouter, LiteLLM and Bifrost is not a feature matrix exercise. The three products answer different questions, and the right one is usually determined by where the gateway runs and what else lives alongside it.

OpenRouter runs as a hosted aggregator. A team sends traffic to its endpoint and OpenRouter fans it out to providers, paying list-price inference and a fee on credit purchases. Its API reference and provider routing documentation describe a managed service that gives one API across many model providers, with the routing, retries and rate limiting handled on OpenRouter’s side.

LiteLLM runs inside a team’s perimeter as either a Python SDK inside an application or a self-hosted proxy server in front of applications. Its model is that Python developers can import a client and change their provider by changing a config file, or operations teams can run the proxy and hand out keys to applications. Both halves of LiteLLM share the same model list and the same enterprise feature set.

Bifrost runs inside a team’s perimeter as a Go gateway. Its overview documentation describes it as “one interface to 10,000+ models across 25+ providers” with an MCP gateway in the same binary. The code is Apache 2.0, the deployment is a binary or a Docker image, and clustering, SSO, guardrails and audit logs sit in a paid enterprise tier.

The gateways work in production in different starting situations. A small team shipping a consumer AI app may do best with OpenRouter and no gateway of its own. A Python-heavy team with a few hundred requests per second may stay on LiteLLM. A regulated enterprise with MCP tools, agents and compliance obligations usually ends up on a gateway like Bifrost. These are tendencies, not rules, and the sections that follow cover what each one actually does.

How the comparison was scored

The three products were scored against criteria written before the list was filled, chosen to surface the differences that matter once traffic grows.

CriterionWhat was checkedWhy it matters
Where the gateway runsHosted or self-hosted; residency and data pathDecides where prompts and responses travel
Runtime and overheadLanguage, process model, added latencyThe gateway sits in front of every model call
Provider breadthNumber of supported model providers and modelsDetermines ability to shift traffic for cost or resilience
GovernanceVirtual keys, budget hierarchy, rate limits, SSOWho can spend what on which model
MCP and agent trafficTool discovery, filtering per key, agent modeAgents produce a growing share of gateway traffic
Observability and guardrailsTracing, metrics, PII and content checksRequired for incident response and compliance
Enterprise boundaryWhat needs a paid licenseReal cost of a compliant deployment

A gateway that scores well on all seven does not exist; the three products are designed to be strong on different rows.

OpenRouter vs LiteLLM vs Bifrost at a glance

DimensionOpenRouterLiteLLMBifrost
Where it runsHosted, third-party endpointSelf-hosted Python SDK or proxySelf-hosted Go binary, Docker or SDK
LicenseProprietary hosted serviceMIT core, enterprise license keyApache 2.0 core, enterprise tier
Provider breadth400+ models across many providers100+ providers25+ providers, 10,000+ models
RuntimeManaged cloudPython Uvicorn workers; opt-in Rust core in betaCompiled Go with per-provider goroutine pools
Published performanceLatency depends on provider and OpenRouter hop; no comparable microbenchmark8 ms P95 latency at 1,000 RPS against a fake OpenAI endpoint11 µs added overhead at 5,000 RPS on t3.xlarge; 424 RPS vs 44.84 RPS in the same Python comparison
GovernanceAPI keys and limits; account-level spendVirtual keys, team, organization, product budgets; some enterpriseVirtual keys with model, provider, tool, guardrail and budget binding
MCP gatewayNoYes, by key, team or organizationYes, deny-by-default per key; code mode; agent mode
ObservabilityAnalytics dashboard; API usage logsLogging callbacks, Prometheus, spend logsOpenTelemetry GenAI spans, Prometheus, request logs
Cost modelList-price inference plus 5.5 to 8 percent fee on credit top-ups; BYOK fee above $25k/monthSelf-hosted inference plus infra; enterprise licenseSelf-hosted inference plus infra; enterprise license

1. Bifrost: production self-hosted

Bifrost, the open-source Go gateway from Maxim AI, is the strongest option for teams that want model, MCP and agent traffic governed by one process on their own infrastructure. The design choice that drives everything else is that the gateway is compiled, concurrent and stateful in memory; the state model that LiteLLM runs in Redis and Postgres is in-process in Bifrost, with asynchronous writes to a backing store.

Runtime and overhead

The t3.xlarge benchmark page breaks down the numbers that follow. On an AWS t3.xlarge at 5,000 requests per second, Bifrost adds 11 microseconds of overhead per request, with 100 percent success. The breakdown includes 1.67 microseconds of queue wait, 10 nanoseconds of weighted key selection, and a few microseconds each for message formatting and request assembly, with peak memory at about 3.3 GB of the instance’s 16 GB. The same comparison puts Bifrost’s throughput at 424 requests per second against 44.84 requests per second for a Python-based alternative, memory at 120 MB against 372 MB, and a success rate of 100 percent against 88.78 percent.

The concurrency design is per-provider goroutine pools, which means a slow or failing provider does not block traffic bound for another. The retries and fallbacks feature runs weighted load balancing across providers at the same layer, which is what resilience looks like under traffic.

Governance, MCP and agents

Bifrost’s virtual keys bind a caller’s identity from the IdP to the four objects a governance tool has to hold together: which models and providers the caller may use, which budgets apply, which MCP tools the caller may call and which guardrails run on their traffic. The MCP documentation describes tool filtering as deny-by-default, which maps cleanly to the specification’s view of tools as code execution.

Code mode addresses the token cost of large tool catalogs. In Bifrost’s published benchmark, input tokens fell 58.2 percent with 96 tools across six servers and 92.8 percent with 508 tools across 16 servers, cutting the 508-tool run from $377 to $29, a 92 percent reduction. Agent mode runs the tool loop with explicit allow-lists and a maximum depth of ten.

Semantic caching and observability

Semantic caching adds exact-match and similarity-based caching to the request path, which cuts cost on repeated queries without changing applications. Observability emits OpenTelemetry GenAI spans and Prometheus metrics for every request. The enterprise tier adds HMAC-signed audit logs with SIEM export, SOC 2 Type II, ISO 27001, HIPAA and GDPR, and extends to employee laptops through Bifrost Edge.

Evidence qualifications: the performance numbers above are from Bifrost’s own published benchmark on an AWS t3.xlarge; they should not be generalised to every workload. Token-cost claims for code mode are from the same vendor-run benchmark.

Best for: teams running production AI traffic that want models, tools, agents, budgets, guardrails and audit in one gateway they can self-host, with Apache 2.0 code and a compiled runtime.

2. LiteLLM: Python-first proxy

LiteLLM is the strongest option for teams whose applications are already written in Python and that want the gateway to feel like a library import. Its production deployment guide sizes the proxy at one Uvicorn worker per pod with 1 vCPU and 4Gi of memory, plus Redis once more than one instance runs and Postgres for spend and config. Scale means more pods and more database connections.

LiteLLM’s provider breadth is a genuine strength. Its documentation lists support for 100+ providers, and the Python SDK makes calling any of them a one-line change in an application. Virtual keys govern model access, spend and rate limits at the key, user, team and organization level. The proxy’s MCP gateway manages per-key, team and organization permissions, with OAuth, client credentials and RFC 8693 token exchange for upstream servers. Enterprise features including SSO, audit logs, guardrails and advanced analytics sit behind a license key on the same Docker image.

Two things are worth knowing about LiteLLM at scale. First, its state model requires Redis and Postgres as the proxy fleet grows, which is normal for a Python service but has to be factored into the compliance envelope. Second, its benchmark page reports 8 ms P95 latency at 1,000 RPS against a fake OpenAI endpoint, which is a different test regime from Bifrost’s t3.xlarge run; comparing the two directly requires normalising for RPS, hardware and what the “fake” endpoint omits.

LiteLLM’s position in the market is covered more fully in the companion piece on LiteLLM alternatives, which maps the Python-proxy trade-off onto what teams typically move to.

Best for: teams already committed to Python runtimes that want a proxy and SDK from the same project, and that are comfortable running Postgres and Redis alongside it.

3. OpenRouter: hosted aggregator

OpenRouter is the strongest option for small teams and prototypes that want one API, pay-as-you-go pricing and nothing to operate. Its pricing documentation describes list-price inference with no markup on token usage, a fee on credit purchases (5.5 percent on Standard, 8 percent on Business, 5 percent for crypto) and bring-your-own-key usage that is free up to $25,000 of list-price inference a month, then 5 percent. For a team that would otherwise manage keys to OpenAI, Anthropic, Google and a few open-weights providers, OpenRouter removes a weekend of setup.

Its provider routing documentation covers the useful part of the service: a configurable order of preferences for which provider serves which model, retries across providers and fallbacks between model tiers. The feature reduces the cost of a provider outage from “rewrite your application” to “wait.” The privacy page says prompts and completions are not stored by default; upstream providers have their own retention policies, which OpenRouter lets a team filter with zero-data-retention settings.

The constraints are structural. OpenRouter is not self-hostable, so a team cannot keep prompts inside its perimeter. Governance runs at the account level and through API keys; identity-based policy, MCP governance and request-path guardrails are not part of its documented surface. The fee on spend becomes material quickly, and the Business plan’s 8 percent on credit top-ups is a real cost at enterprise volumes.

Teams that outgrow the hosted model usually keep OpenRouter and put a self-hosted gateway in front. Bifrost’s OpenRouter provider page documents the integration, which means a team can move incrementally without a flag day. The broader map of post-OpenRouter choices is in the companion piece on OpenRouter alternatives.

Best for: small teams, prototypes and side projects that want one hosted API and are comfortable with a third party in the data path.

Which workload each one fits

The decision rarely comes down to “which gateway is best.” It comes down to which workload matches which starting point.

Starting pointFirst gateway to evaluateReason
Production, regulated, self-hosted, multi-agentBifrostOne process for models, MCP and agents; Apache 2.0 and enterprise tier
Python application, modest trafficLiteLLMSDK and proxy from one project; easy drop-in
Prototype, small team, no operations headcountOpenRouterHosted, pay-as-you-go, one API across many providers
Enterprise migration off OpenRouterBifrost with OpenRouter as a providerMove incrementally without changing applications
Python-heavy team with growing agentsBifrost in front of LiteLLMKeep the SDK, add agent and MCP governance

The migration pattern is worth stating directly because it is the most common path. Teams that start on OpenRouter outgrow the fee on spend and the lack of identity-based policy. They put a self-hosted gateway, usually Bifrost or LiteLLM, in front and register OpenRouter as one provider behind it. They then move individual models to direct provider keys as volume justifies it, retiring OpenRouter from the critical path for those models and keeping it for the tail. Nothing in that sequence requires a flag day.

Verdict

Bifrost is the strongest option for production self-hosted deployments because one process governs models, MCP and agents, its published benchmark reports 11 microseconds of added overhead per request at 5,000 RPS and 424 requests per second against 44.84 for a Python alternative, and its enterprise tier closes the audit and compliance gap. LiteLLM is the strongest option for Python-first teams that want an SDK and a proxy from one project. OpenRouter is the strongest option for small teams and prototypes that want a hosted API and no operational work.

The trio is not mutually exclusive. A production enterprise can run Bifrost as its control plane and register OpenRouter as a provider for long-tail models. A Python-heavy team can keep LiteLLM as an SDK inside its application and put Bifrost in front for governance. The pattern that does not work is the inverse: running OpenRouter inside a regulated perimeter without a gateway in front, or using LiteLLM’s proxy as an identity-based governance tool without its enterprise license.

Limits of this comparison

Capabilities and pricing come from vendor documentation read in November 2026. No product was deployed or load-tested for this piece, and no vendor benchmark was reproduced. Bifrost’s throughput, overhead and code-mode numbers are from Maxim AI’s own published t3.xlarge run; LiteLLM’s 8 ms P95 number is from a different test regime; OpenRouter’s documentation does not publish a comparable microbenchmark. OpenRouter’s fee structure is dated October 2026 and may change. Pricing beyond those listed figures was not compared.

Sources

  1. Bifrost source repository (GitHub, Apache 2.0)
  2. Bifrost docs: overview
  3. Bifrost docs: t3.xlarge benchmark
  4. Bifrost docs: virtual keys
  5. Bifrost docs: budgets and limits
  6. Bifrost docs: MCP overview
  7. Bifrost docs: MCP code mode
  8. Bifrost docs: OpenRouter provider
  9. Bifrost docs: retries and fallbacks
  10. Bifrost docs: built-in observability
  11. Bifrost docs: semantic caching
  12. Maxim AI: Bifrost AI gateway
  13. Maxim AI: Bifrost MCP gateway
  14. Maxim AI: Bifrost governance
  15. LiteLLM: proxy server overview
  16. LiteLLM: production deployment
  17. LiteLLM: virtual keys
  18. LiteLLM: enterprise features
  19. LiteLLM: MCP gateway
  20. OpenRouter API reference
  21. OpenRouter: pricing
  22. OpenRouter: provider routing
  23. OpenRouter: privacy

Questions readers ask

What is the difference between OpenRouter, LiteLLM and Bifrost?

OpenRouter is a hosted aggregator; a team sends traffic to it and it fans out to providers, with pay-as-you-go billing for inference and a fee on credit purchases. LiteLLM is a Python SDK and self-hosted proxy with provider breadth and virtual keys. Bifrost is a Go gateway that runs on a team's own infrastructure with MCP, agent features and enterprise governance in one process.

Which one is the fastest?

Bifrost's own benchmark on an AWS t3.xlarge reports 11 microseconds of added overhead per request at 5,000 requests per second, with 100 percent success. LiteLLM's documentation reports 8 ms P95 latency at 1,000 RPS against a fake OpenAI endpoint in its own benchmark. OpenRouter adds the latency of an external hop to a hosted aggregator; its documentation does not publish a comparable microbenchmark. Each benchmark was run by the vendor and is scoped to its own test.

Which one is self-hostable?

Bifrost (Apache 2.0) and LiteLLM (MIT outside its enterprise directory) are both open source and self-hostable. OpenRouter is a hosted service only.

Which one is best for agents?

Bifrost ships an MCP gateway and agent mode in the same process, which means model, tool and subagent permissions can sit on one key. LiteLLM has an MCP gateway with permissions by key, team or organization. OpenRouter focuses on model routing and does not provide an MCP gateway.

Can a team use all three?

Yes, and some do. Bifrost as the control plane, with LiteLLM retired or kept for its SDK, and OpenRouter registered as one provider behind Bifrost, is a common migration path for teams that started on a hosted aggregator.

More comparisons