Coding-agent productivity needs a systems denominator
Coding-agent activity is easy to count. Useful delivery requires accounting for review, infrastructure, retries and the quality of accepted changes.
Topic
Infrastructure
Fact check: LiteLLM's Rust benchmark claims, tested against its own repo files
LiteLLM's homepage and benchmark blog credit its Rust gateway with about 7x lower overhead, 9x less memory and about 4.5x more requests per dollar. Fifteen claims, set against the CSV and JSON files LiteLLM published, with neutral verdicts, including the ones that check out.
LiteLLM's Rust gateway benchmark, checked against LiteLLM's own data
LiteLLM says its Rust gateway has about 7x lower overhead, 9x less memory and 4.5x more requests per dollar than the next-closest gateway. LiteLLM's own benchmark files show a latency and throughput near-tie, a mislabeled widest-pair memory chart, a modeled cost and a config gap, plus Rust leads on CPU and memory.
LiteLLM vs Bifrost latency: tracing the 7x overhead figure
LiteLLM says its Rust gateway has roughly 7x lower overhead than Bifrost. The concurrency sweep in LiteLLM's own benchmark repo shows Bifrost lower at concurrency 1 and 64, Rust lower at 16, a tie at 4, and throughput gaps under 1%. The headline number comes from a separate file with no stated concurrency.
Logging on vs logging off: the configuration gap in LiteLLM's benchmark
LiteLLM's benchmark blog says no gateway logged or persisted anything. The Bifrost v1.6.4 config it published never sets enable_logging or a log store, and the defaults then switch on logging, a SQLite store and built-in plugins. That is inferred from v2.4.0 code, not confirmed on v1.6.4.
A Rust beta vs an old Bifrost build: the version problem in software benchmarking
LiteLLM's Rust gateway benchmark compared an early Rust beta, which its own blog says is still landing streaming, against Bifrost v1.6.4, an old build. The homepage never names the version. Why version parity decides what a benchmark can claim, what the record shows, and a checklist for readers.
AI energy per prompt estimates span 2,300-fold, but typical text prompts agree at 0.24 to 0.42 Wh
The Frontier Wire compared every per-prompt energy and water figure in its dataset. The extremes differ 2,300-fold for energy and over 1,600-fold for water, but four independent estimates for an ordinary text prompt sit within a factor of 1.75. Scope, request length and modality explain the rest.
AI water use per kWh fell at six big tech firms, yet total consumption rose at all six
An analysis of company sustainability reports in The Frontier Wire's AI water and energy dataset. Every one of the six companies that publish two years of water and electricity used less water per kilowatt-hour. None used less water. Together they added about 17.7 million cubic meters in a year.
How to choose an API management solution: a weighted framework
API management products share a feature list and differ in where their planes run, how their policies port and what they can meter. A weighted criteria table, a proof-of-concept plan and a cost model, grounded in the OpenAPI and OAuth specs and OWASP's 2023 API risks, plus the new question of model traffic.
The Model Context Protocol, explained from the specification
MCP is now stateless. Revision 2026-07-28 removed the initialize handshake and protocol sessions, deprecated sampling and roots, and moved server-to-client requests into retried calls. A section-by-section guide to the current specification, its security rules, its registry and SDKs, and how teams run it in production.
Prompt caching in practice: breakpoints, TTLs and cost math
Every major model API now discounts a repeated prompt prefix, but the rules differ on markers, minimum lengths, lifetimes and write surcharges. A provider-by-provider implementation guide with current prices, the usage fields that prove a hit, worked cost examples and the gateway settings that keep a cache warm.
Rate limiting: a practical guide to algorithms, 429s and LLM limits
Six algorithms, four places to enforce them, one status code and a draft standard for telling clients how much quota is left. A working reference for rate limiting HTTP APIs, with the request and token limits that OpenAI, Anthropic and Google apply to model traffic.
A2A vs MCP: how the two agent protocols divide the work
Both protocols now sit under the Linux Foundation's Agentic AI Foundation, and both shipped major revisions in 2026. Read side by side, the specifications draw a clean line: MCP gives one agent depth, A2A gives a system of agents reach.
Prompt caching vs semantic caching: what each one saves
Provider prompt caches cut the price of a repeated prefix by 90 percent or more but leave output tokens untouched. Semantic caches remove whole calls but can return the wrong answer. The mechanics, prices and failure modes of both, with a worked cost example.
Self-hosted LLMs: how to run a local LLM, from laptop to GPU cluster
Running a model on your own hardware is now a configuration problem more than a research one. The hard parts are sizing GPU memory, choosing between Ollama, llama.cpp, LM Studio, vLLM and SGLang, reading the licence, and putting a production API in front of a server that was built for one user.
Top 5 AI gateways for multimodal LLM workloads in 2026
A multimodal LLM stack sends images, audio, video and files as well as text. Five gateways, measured endpoint by endpoint against the same published criteria, and which kind of team each one suits.
Top 5 AI gateways for routing between vLLM, Ollama and cloud LLMs in 2026
Most teams now run some models on their own GPUs and rent the rest. Five self-hostable gateways that route between vLLM, Ollama and hosted APIs, compared on how they reach local servers, how they decide where a request goes, and what happens when a local box falls over.
Top 5 AI gateways for air-gapped and in-VPC LLM deployments in 2026
An on-premise LLM stack is only as private as the gateway in front of it. Five self-hosted AI gateways, read at the level of what each one fetches from the internet, how it installs without a connection, and how it routes to vLLM, Ollama and private cloud endpoints.
The 10 best agent orchestration platforms for enterprise in 2026
An enterprise AI agent platform is where agents run, get an identity, reach tools and get audited. Ten managed options, from the three hyperscalers to the SaaS suites and an automation tool, compared on published criteria with documented prices.
Top 10 AI gateways in 2026, ranked on production criteria
An AI gateway now sits in front of most production model traffic. Ten options, from open-source binaries to cloud API platforms and managed routers, measured against ten published criteria before any ranking.
Top 10 open-source AI gateways for self-hosting in 2026
An open-source AI gateway is only as open as its license file and its paid tier allow. Ten self-hosted LLM gateways, read at the level of the LICENSE file, the dependency list and the release history, with criteria published before the ranking.
LLM cost management: budgets, rate limits and virtual keys at the gateway
Output tokens cost five to six times more than input tokens, reasoning tokens bill as output, and shared API keys erase the link between a charge and its owner. This is how teams attribute and cap LLM spend, and what gateway-level governance in the open-source Bifrost gateway does about it.
LLM failover and load balancing: keeping AI apps up when providers fail
Model APIs fail in documented, predictable ways. The status pages, error codes and rate-limit rules published by OpenAI and Anthropic show which failures a retry fixes, which need a different provider, and which no amount of rerouting will solve.
AI gateway vs API gateway: what changes when the traffic is tokens
API gateways were built for short request-response calls priced per request. Model traffic is priced per token, streamed for minutes and shaped differently by every provider. A mechanism-by-mechanism comparison, grounded in AWS, Kong, NGINX, OpenAI and Anthropic documentation, with Bifrost as the worked example.
What an AI gateway does, and what it costs to run one
Most teams that use more than one model provider end up putting a proxy in front of all of them. This is what that proxy does, where the main products differ, and a detailed look at one of them, Bifrost, based on its documentation and a local install.