Archive
Output tokens cost five to six times more than input tokens, reasoning tokens bill as output, and shared API keys erase the link between a charge and its owner. This is how teams attribute and cap LLM spend, and what gateway-level governance in the open-source Bifrost gateway does about it.
Model APIs fail in documented, predictable ways. The status pages, error codes and rate-limit rules published by OpenAI and Anthropic show which failures a retry fixes, which need a different provider, and which no amount of rerouting will solve.
API gateways were built for short request-response calls priced per request. Model traffic is priced per token, streamed for minutes and shaped differently by every provider. A mechanism-by-mechanism comparison, grounded in AWS, Kong, NGINX, OpenAI and Anthropic documentation, with Bifrost as the worked example.
TypeSafe's Jev returns typed answers with probabilities instead of text. The idea borrows Kahneman's System 1, a calibration objective called RLCD, and a strong claim about hallucination. The concepts, the evidence behind them, and where each one stops.
Routing, gating, judging and choosing an agent's next step are decisions, not writing. Splitting decision from generation is the right architecture, and TypeSafe's Jev is the first product built for it. Trust will come from independently measured calibration, not a promise that it cannot hallucinate.
TypeSafe AI came out of stealth on September 15 with $40 million in seed funding and Jev, a model that answers typed questions about a block of text with probabilities instead of prose. The pricing is unusual, the speed claims are large, and the first independent tests are mixed.
Models that think before they answer are billed for every token of that thinking, whether or not you see it. The controls have moved from token budgets to effort levels, and the price of a request now depends on how hard the question is.
The idea that a model can get better by thinking longer, rather than by being bigger, reshaped how labs spend their money. The evidence behind it, method by method, and the limits the papers themselves report.
Most teams that use more than one model provider end up putting a proxy in front of all of them. This is what that proxy does, where the main products differ, and a detailed look at one of them, Bifrost, based on its documentation and a local install.
Every evaluation framework claims to measure quality. The differences show up in who can write the test, whether a trajectory can fail, and what happens to a bad score in production. Here are the criteria, the research that justifies them, and nine current tools read against them.
The word implies a glitch in an otherwise reliable mind. The published evidence says something plainer: models are trained and graded to guess, and the remedy depends on saying so.
Most flagship open-weight releases of the past year ship under Apache 2.0 or MIT. The exceptions carry user caps, revenue gates, prohibited-use policies and remote kill switches. This survey reads the licence texts, not the launch posts, and maps what each one actually permits.
Every frontier API now takes a million tokens, and every published long-context study still finds accuracy falling well before the window is full. Retrieval fails differently, earlier and more visibly. The failure you can tolerate should decide, and current pricing tells you what each choice costs.
The model card was proposed as a one-to-two-page disclosure. Current cards range from a 135-page system card to a benchmark table with a licence. Reading one well means checking it against the original template, the EU AI Act and the Stanford transparency index.