Section
Research
Briefings on the papers, benchmarks and techniques that move the field, explained without the jargon.
Fact check: LiteLLM's Rust benchmark claims, tested against its own repo files
LiteLLM's homepage and benchmark blog credit its Rust gateway with about 7x lower overhead, 9x less memory and about 4.5x more requests per dollar. Fifteen claims, set against the CSV and JSON files LiteLLM published, with neutral verdicts, including the ones that check out.
LiteLLM vs Bifrost latency: tracing the 7x overhead figure
LiteLLM says its Rust gateway has roughly 7x lower overhead than Bifrost. The concurrency sweep in LiteLLM's own benchmark repo shows Bifrost lower at concurrency 1 and 64, Rust lower at 16, a tie at 4, and throughput gaps under 1%. The headline number comes from a separate file with no stated concurrency.
FlashProve and the limits of a cryptographic AI receipt
FlashProve makes model execution a checkable claim. Its research prototype also exposes the limits of what an AI receipt can guarantee.
Logging on vs logging off: the configuration gap in LiteLLM's benchmark
LiteLLM's benchmark blog says no gateway logged or persisted anything. The Bifrost v1.6.4 config it published never sets enable_logging or a log store, and the defaults then switch on logging, a SQLite store and built-in plugins. That is inferred from v2.4.0 code, not confirmed on v1.6.4.
A Rust beta vs an old Bifrost build: the version problem in software benchmarking
LiteLLM's Rust gateway benchmark compared an early Rust beta, which its own blog says is still landing streaming, against Bifrost v1.6.4, an old build. The homepage never names the version. Why version parity decides what a benchmark can claim, what the record shows, and a checklist for readers.
Inside Google’s building blocks for agent-assisted wet labs
Google’s lab studies separate physical execution, molecular search and experimental validation. The distinctions explain what the agents achieved.
Attribute-based access control (ABAC): a practical guide
NIST defines attribute-based access control as decisions made from attributes of the subject, the object and the environment. The architecture behind it, how it differs from RBAC and Google's Zanzibar-style ReBAC, the policy languages that implement it, and how to apply it to models, agent tools, MCP servers and RAG.
Claude usage: plan limits, how to check them and API tracking
Claude usage is measured three different ways depending on how a request signs in. Subscriptions run on five-hour and weekly windows, API keys run on rate limits and spend caps, and usage-based Enterprise seats are billed per token. Where each meter lives and what drains it, per Anthropic's pages on 21 October 2026.
The sleep framework puts forgetting at the centre of AI learning
Sleep consolidates temporary information into model parameters. Its experiments clarify what retention means and what the extra computation buys.
How to choose an API management solution: a weighted framework
API management products share a feature list and differ in where their planes run, how their policies port and what they can meter. A weighted criteria table, a proof-of-concept plan and a cost model, grounded in the OpenAPI and OAuth specs and OWASP's 2023 API risks, plus the new question of model traffic.
The Model Context Protocol, explained from the specification
MCP is now stateless. Revision 2026-07-28 removed the initialize handshake and protocol sessions, deprecated sampling and roots, and moved server-to-client requests into retried calls. A section-by-section guide to the current specification, its security rules, its registry and SDKs, and how teams run it in production.
Agent benchmarks are now 44% of new AI benchmarks, and most are too small to rank close models
An analysis of 352 sized benchmarks in The Frontier Wire's AI benchmarks dataset finds agent benchmarks are about 4.5 times smaller than the rest. That makes many leaderboard gaps statistical noise. The data, the method, and a table of the smallest score gap worth taking seriously at each benchmark size.
Prompt caching in practice: breakpoints, TTLs and cost math
Every major model API now discounts a repeated prompt prefix, but the rules differ on markers, minimum lengths, lifetimes and write surcharges. A provider-by-provider implementation guide with current prices, the usage fields that prove a hit, worked cost examples and the gateway settings that keep a cache warm.
Rate limiting: a practical guide to algorithms, 429s and LLM limits
Six algorithms, four places to enforce them, one status code and a draft standard for telling clients how much quota is left. A working reference for rate limiting HTTP APIs, with the request and token limits that OpenAI, Anthropic and Google apply to model traffic.
AI energy per prompt estimates span 2,300-fold, but typical text prompts agree at 0.24 to 0.42 Wh
The Frontier Wire compared every per-prompt energy and water figure in its dataset. The extremes differ 2,300-fold for energy and over 1,600-fold for water, but four independent estimates for an ordinary text prompt sit within a factor of 1.75. Scope, request length and modality explain the rest.
Shadow AI monitoring: a practical guide to detection, inventory and policy
Shadow AI detection is a data problem before it is a blocking problem. The signal sources that reveal unapproved AI tools, what to log and how long to keep it, how to build an AI inventory that NIST and ISO 42001 would recognise, and how to roll monitoring out through MDM without breaking employee privacy law.
User-managed access and rate limiting: a technical guide
User-Managed Access lets a resource owner decide who else may reach their data, through permission tickets and requesting party tokens. Every identity that grant creates is also a quota key. The protocol, the HTTP signals for throttling, and how both apply to LLM APIs and agents acting for users.
AI water use per kWh fell at six big tech firms, yet total consumption rose at all six
An analysis of company sustainability reports in The Frontier Wire's AI water and energy dataset. Every one of the six companies that publish two years of water and electricity used less water per kilowatt-hour. None used less water. Together they added about 17.7 million cubic meters in a year.
A2A vs MCP: how the two agent protocols divide the work
Both protocols now sit under the Linux Foundation's Agentic AI Foundation, and both shipped major revisions in 2026. Read side by side, the specifications draw a clean line: MCP gives one agent depth, A2A gives a system of agents reach.
Claude Code security: permissions, sandboxing and the scanner
The phrase now means two things: Anthropic's Claude Security scanner, and the job of keeping a coding agent with shell access from doing harm. Permission modes, the sandbox, managed settings, MCP policy and gateway routing, mapped to the incidents that shaped them.
LLM observability: what to capture, trace and alert on in production
A practical explainer on LLM observability: the per-call record, time to first token, OpenTelemetry GenAI conventions, trace structure for agents, sampling and PII redaction, Prometheus alert thresholds, feedback joins and evals on production traffic, plus the open-source llm-observability-stack agent skill.
OWASP LLM Top 10 for 2026: the ten risks and where to enforce each
The 2026 edition of the OWASP Top 10 for LLM applications is the first built partly on incident data. It reorders the list, renames one entry and hands agent risks to a separate Agentic Top 10. Each of the ten risks, with documented examples and the layer that can actually enforce its controls.
Prompt caching vs semantic caching: what each one saves
Provider prompt caches cut the price of a repeated prefix by 90 percent or more but leave output tokens untouched. Semantic caches remove whole calls but can return the wrong answer. The mechanics, prices and failure modes of both, with a worked cost example.
System One models, explained: typed answers and calibration
TypeSafe's Jev returns typed answers with probabilities instead of text. The idea borrows Kahneman's System 1, a calibration objective called RLCD, and a strong claim about hallucination. The concepts, the evidence behind them, and where each one stops.
Prompt injection: how the attacks work and which defenses hold
Every major lab, OWASP and the UK's National Cyber Security Centre now say the same thing about prompt injection. Models cannot reliably tell instructions from data, filters miss adaptive attacks, and the defenses that work limit what a manipulated model is able to do.
Self-hosted LLMs: how to run a local LLM, from laptop to GPU cluster
Running a model on your own hardware is now a configuration problem more than a research one. The hard parts are sizing GPU memory, choosing between Ollama, llama.cpp, LM Studio, vLLM and SGLang, reading the licence, and putting a production API in front of a server that was built for one user.
Test-time compute, explained: the second scaling lever
The idea that a model can get better by thinking longer, rather than by being bigger, reshaped how labs spend their money. The evidence behind it, method by method, and the limits the papers themselves report.
Shadow AI: what it is, how widespread it is, and how to detect it
Two thirds of AI users reach AI services through personal accounts on corporate devices, and shadow AI now appears in 43% of the AI security incidents IBM studied. The risks, the four detection layers, the acceptable use policy and the controls that turn unsanctioned use into governed use.
LLM cost management: budgets, rate limits and virtual keys at the gateway
Output tokens cost five to six times more than input tokens, reasoning tokens bill as output, and shared API keys erase the link between a charge and its owner. This is how teams attribute and cap LLM spend, and what gateway-level governance in the open-source Bifrost gateway does about it.
Reading a model card like an auditor
The model card was proposed as a one-to-two-page disclosure. Current cards range from a 135-page system card to a benchmark table with a licence. Reading one well means checking it against the original template, the EU AI Act and the Stanford transparency index.
LLM failover and load balancing: keeping AI apps up when providers fail
Model APIs fail in documented, predictable ways. The status pages, error codes and rate-limit rules published by OpenAI and Anthropic show which failures a retry fixes, which need a different provider, and which no amount of rerouting will solve.
AI gateway vs API gateway: what changes when the traffic is tokens
API gateways were built for short request-response calls priced per request. Model traffic is priced per token, streamed for minutes and shaped differently by every provider. A mechanism-by-mechanism comparison, grounded in AWS, Kong, NGINX, OpenAI and Anthropic documentation, with Bifrost as the worked example.