Bifrost reworks /v1/decisions into one OpenAI-shaped API for decision models

Bifrost, the open-source AI gateway from Maxim AI, rebuilt its decision-model endpoint in version 2.3.0, released on October 13, 2026. POST /v1/decisions now takes the same request shape as OpenAI’s Decisions API, reaches native decision models from eight providers, and refuses by default to fake a decision with a chat model. The release also adds billing fields that price a decision separately from a chat call, and lets the gateway’s complexity router use a decision model to sort incoming prompts. The company announced the expanded endpoint on October 16. A month after TypeSafe put the first commercial System One model on sale, decision models have become a small API category with several vendors, two request dialects and input-only pricing.

TL;DR

• Bifrost v2.3.0 (October 13) moves /v1/decisions to OpenAI’s shape: an input, an ordered questions list, and question types predicate, choice and score. The older map form is deprecated but still served.
• The endpoint now reaches native decision models from TypeSafe, OpenAI, OpenRouter, Perplexity, Cloudflare, Azure, Databricks and Ollama. Azure and Databricks arrived in v2.3.1 on October 19.
• Emulating a decision through a chat model’s Responses API is now opt-in. A decision request to a chat model returns unsupported_operation unless the operator sets convert_decision_to_responses.
• New input_cost_per_token_decisions and output_cost_per_token_decisions fields price decisions apart from chat, which matters for models such as gpt-6-luna that are billed differently on each endpoint.
• The complexity router can now classify prompts with a decision model, TypeSafe’s Jev by default.

What shipped, and when

Bifrost first added /v1/decisions in v2.2.2 on September 23, eight days after TypeSafe launched Jev. That release added a typesafe provider, a /typesafe route for TypeSafe’s own SDKs, and a fallback: if a provider had no decision endpoint, the gateway answered “through forced tool-calling on their chat model.” At that point TypeSafe was the only native backend. The September announcement said the feature shipped “for both OSS and Enterprise.”

The October releases changed the endpoint in five ways. The v2.3.0 changelog lists them as separate items:

The October 16 announcement, by Madhu Shantan, describes “decision models from eight providers, plus any self-hosted decision model you add as a custom provider.” Azure and Databricks are itemised in the v2.3.1 changelog, dated October 19. None of these items carries an enterprise label.

A new API shape, in two dialects

A decision model, which TypeSafe calls a System One model, does not write text. It reads an input and answers typed questions with probabilities: how likely a statement is to be true, which of a fixed set of options fits, or where the input sits on an ordered scale. This publication covered the idea in System One models, explained and argued for the split between deciding and generating in System One models vs LLMs. TypeSafe’s launch is reported in TypeSafe launches Jev.

What has changed since September is the number of vendors. By late October, at least seven companies documented a decision endpoint, and they split into two dialects.

The first is TypeSafe’s System One format. According to TypeSafe’s API reference, a request to POST /v1/systemone carries a state (the text or JSON being judged) and a map of named questions. Yes/no questions are called noul, and answers come back keyed by name. Most of the market copied it. Perplexity’s Decisions API takes state and named noul, choice and score questions. Microsoft’s Microsoft-Decision-1 page says the model “accepts a state value and one or more typed questions” and that the pattern “is similar to a system-one API.” Databricks’ ai_decide function, labelled Beta, does the same. Ollama added a /v1/systemone endpoint in version 0.35, announced September 29, and its example uses TypeSafe’s own Python SDK. Cloudflare describes a “Clef extension to the System One API” that adds images.

The second dialect is OpenAI’s. Its Decisions guide says the API is now generally available and that gpt-6-luna is “the only currently supported model.” A request has a model, an input (a string or an ordered array of messages, tool calls and tool outputs) and a questions list. The yes/no type is called predicate, and the response is an answers array that echoes each question’s name. OpenAI allows up to 400 questions per request.

The two dialects ask the same three kinds of question but differ in field names, map versus list, and how answers are keyed. Code written against one will not run against the other without translation.

Prices from TypeSafe’s models page, OpenAI’s Decisions guide, Perplexity’s docs and Cloudflare’s clef-flash page, fetched October 28. Other details from Bifrost’s provider pages.

Every vendor that publishes a decision price charges for input only, because the answer is a few probabilities bounded by the schema, not a stream of generated tokens. For context on what generative models cost per token, see LLM API pricing compared.

The unified request

Bifrost’s own format had followed TypeSafe’s: state plus a map of named questions, with noul for yes/no. In v2.3.0 the gateway moved its unified shape to OpenAI’s dialect instead. The decisions quickstart shows a request with a model, an input and an ordered list of questions, each with a type, an optional name, instructions, and choices or levels:

{
  "model": "typesafe/jev-latest",
  "fallbacks": ["cloudflare/clef-flash", "openai/gpt-6-luna"],
  "input": "Customer message: I was double charged and nobody replied to my two emails.",
  "questions": [
    { "type": "predicate", "name": "is_frustrated", "instructions": "Is the customer frustrated?" },
    { "type": "choice", "name": "category", "instructions": "Pick the ticket category",
      "choices": [{ "value": "billing" }, { "value": "bug" }, { "value": "other" }] }
  ]
}

Answers come back as a list in question order. A predicate answer carries a probability. Choice and score answers carry the selected value plus confidence, probabilities and, for scores, a legend, when the model supplies them. A model that declines a question answers it with "type": "refusal". Switching providers means changing the model string, written as provider/model.

The gateway does the dialect conversion. The TypeSafe provider page says it maps predicate to noul and choices and levels to TypeSafe’s criteria. The OpenAI provider page says structured input goes to OpenAI as JSON text, and a predicate’s criteria is appended to its instructions.

Three routes into the same models

The unified endpoint is one of three routes. According to the quickstart, POST /openai/v1/decisions takes OpenAI’s native request, so OpenAI SDKs work with a changed base URL, and POST /typesafe/v1/systemone takes TypeSafe’s, for its SDK. The changelog says the OpenAI-format route also appears under the /langchain, /litellm and /pydanticai prefixes, relays requests to OpenAI-based providers “unchanged” and rebuilds other providers’ answers “into OpenAI’s response shape.”

The routes are not tied to their namesake vendor. The quickstart says every provider “also answers on /openai/v1/decisions and /typesafe/v1/systemone when you send a provider/model name.” The route sets the payload format; the chosen model sets what the input can contain.

The old Bifrost map form, with state and keyed questions, is deprecated but still served, and is answered in the same form. The gateway treats a body as the map form when questions is an object rather than a list. TypeSafe’s own named-map format remains native on the /typesafe route.

Differences the gateway does not hide

The announcement is plain that “a shared API does not make every model interchangeable. Input capabilities, confidence fields, and classification behavior can differ.” The provider pages list where.

Images. OpenAI, Perplexity, Cloudflare and Ollama read images through the unified endpoint. Ollama takes base64 only and puts every image in one list that all questions see. TypeSafe reads images only by reference, with an id placed in the input, instructions or criteria, and Bifrost adds the TypeSafe-Danger-Zone-Images header TypeSafe requires. The Azure and Databricks paths are text only, and Bifrost rejects an image with a 400 so that a fallback which reads images can take the request.

Strictness. Cloudflare’s Clef returns a 422 for a null input or any field it does not define, according to the Cloudflare provider page. Azure returns its own 400 for unrecognised top-level fields. Databricks’ ai_decide rejects a model field entirely, so the gateway sends none and pins the function version with options.version: "1.0".

Limits. Azure accepts 2 to 255 choice options, 2 to 10 score levels and a 32,768-token context. Perplexity accepts 1 to 10 score levels. Cloudflare’s clef-flash rejects inputs over 24,576 tokens with a 413.

Addressing. Azure maps Microsoft-Decision-1 to a deployment with a key alias. Ollama models need a minimum server version, from 0.35 for nimble to 0.40.0 for laya, per the Ollama provider page, and accept a keep_alive duration.

Versions. The Perplexity provider page lists pplx-decider-v1.1-27b as current. Perplexity’s own docs now say requests naming it “select pplx-decider-v1.2 and bill at its rate.” A gateway passes such alias moves through.

Fallbacks cross these boundaries, as in the example above. An input one model cannot read fails only that attempt; if every attempt fails, the caller gets the first error.

Emulation is now opt-in

The most consequential change in v2.3.0 is a default. Until this release, a decision request that reached a model with no decision endpoint was answered anyway. The September post explains how: the gateway rebuilt the questions as “a single forced emit_decision function tool,” sent it through the model’s Responses API, and mapped the tool-call arguments back to the decision shape. The v2.2.2 changelog adds that probabilities were “normalized to sum to exactly 1.” The same shim served two cases: a caller naming a chat model as the decision model, and a fallback to a chat model after Jev failed. In the second case, the post said, “the caller sees no difference beyond the resolved model name.”

That is the problem the new default addresses. An emulated decision differs from a native one in three ways a caller may care about.

The first is cost. A decision model bills input only, at cents per million tokens; an emulated decision is a full chat call at chat rates. Bifrost’s own September benchmark, which ran a 30-question suite with Jev as the reference, recorded Jev at 920 ms and 2,144 tokens. The same suite emulated on gpt-5.6-luna took 6,763 ms and 3,534 tokens; on claude-opus-5 it took 15,667 ms and 8,203 tokens.

The second is latency: those figures are roughly seven and seventeen times slower.

The third is the meaning of the numbers. The benchmark measured whether emulated answers matched Jev’s, and every model listed scored 29 or 30 out of 30. It did not measure whether the probabilities meant anything. A decision model’s selling point is a probability an application can set a threshold on. A chat model filling in a probabilities field in a tool call is writing numbers, not reporting a calibrated distribution. The compatibility plugin docs now say so directly:

An emulated answer is a chat model’s guess, not a calibrated judgment, and it costs a full chat call.

The docs recommend “keeping this setting off and sending decisions to a model that serves them natively.” With the setting off, a decision request to a model such as openai/gpt-4o fails with HTTP 400, the code unsupported_operation, and a message naming the model and the setting.

Emulation still exists for operators who want it. It is turned on globally with convert_decision_to_responses (in the dashboard, Compatibility, “Emulate Decisions API on non-decision models”) or per request with the header x-bf-compat: ["convert_decision_to_responses"]. When it runs, the response carries extra_fields.converted_request_type: "responses" and is “billed as the chat call it is.” Requests with images or passthrough extra parameters are rejected rather than emulated. The check runs on every attempt, including fallbacks.

The docs flag one interaction. To decide whether to emulate, the gateway checks the catalog’s supports_decisions entry or a built-in list of names, and custom providers have no catalog entries. With emulation on, a self-hosted Laya or Nimble server “is emulated through a Responses API it may not serve instead of answered natively,” including for router classification.

Pricing decisions on their own rates

OpenAI’s pricing for gpt-6-luna shows why a gateway needs separate decision rates. Its Decisions guide says uncached input on /v1/decisions costs $0.10 per million tokens, “output tokens are not charged,” and “other requests using gpt-6-luna follow the applicable model and processing-tier pricing.” The same model has two price sheets, depending on which endpoint is called.

Bifrost’s catalog stores one row per model, so the Decision Pricing Rates item adds two fields to that row. According to the custom pricing docs, input_cost_per_token_decisions and output_cost_per_token_decisions are flat across the context window and “always win” over the standard rates. The standard rates are used only when no decision rate is set. The changelog also notes that decision requests on variable-price models “no longer bill a negative cost.”

decisions is now a request type in pricing overrides and in routing rules written in CEL, the Common Expression Language the gateway uses for rule conditions. Several providers need an override to be billed at all. The Azure and Databricks pages say decisions are priced at $0 until an operator adds one, and the self-hosted guide says the same for custom providers. OpenRouter is the exception: Bifrost reads OpenRouter’s billed amount from usage.cost and uses that for logs and budgets.

Output rates are not always zero in practice. The self-hosted guide notes that Laya reports zero output tokens, while Nimble reports one output token per question plus one.

Self-hosted models and a TypeSafe base

The other way to reach a decision model is to run one. v2.3.0 lets a custom provider use base_provider_type: "typesafe", so any server speaking TypeSafe’s System One API can sit behind the gateway with its own keys and request_path_overrides. The self-hosted guide documents two: Laya, run as a local Python server with english, multilingual and typed-decisions checkpoints, and Bespoke Labs’ Nimble, served with SGLang on Modal. Both are text only.

Ollama is the simpler local route. The Ollama blog lists nimble, a 9B model from Bespoke Labs, and tev1, an experimental 4B model from Together AI. Bifrost’s Ollama provider calls /v1/systemone directly and needs no key for a local server. For how Bifrost and other gateways route between local and hosted models more generally, see LLM routers for vLLM, Ollama and cloud models.

Decision models inside the router

The last v2.3.0 item puts a decision model to work inside the gateway itself. Bifrost’s complexity router sorts each request into a Simple, Medium or Complex tier. Routing rules can then match complexity_tier == "COMPLEX" and send that traffic to a stronger model. Until this release the router classified with a semantic classifier, which embeds the request and matches it against reference phrases.

The decision-model classifier asks one question per request: which tier best describes the complexity of the latest user request. It sends each tier’s definition, signals and examples with the question, all editable. TypeSafe Jev (jev-latest) is the default. The choosing-a-model table lists every native provider above, plus Laya and Nimble, as alternatives. It can run as the primary classifier or as the semantic classifier’s fallback. Only user turns are sent; decision.previous_message_count defaults to 1.

The classifier sits on the request path. The docs say decision.timeout defaults to 1.5 seconds, that Jev “usually answers in 600-800 ms,” and that a timeout leaves the request unclassified. Cold starts matter: clef-flash takes four to six seconds cold, and Nimble on Modal “takes minutes to wake.” Classification calls count toward the request’s cost and budgets under the model that answered, which the announcement describes as making “the classification step part of the operational picture.”

The Bifrost team published its own router benchmark on October 14. On RouterArena’s everyday questions, the benchmark post reports that routing with Jev solved 74.3 percent against 77.0 percent for always using Claude Opus 5.5, at $34.21 against $53.45. On Terminal-Bench 2.1 coding tasks run through Claude Code, with results reported out of 77, it reports 73 solved with Jev routing against 74 for Opus, at $0.282 against $0.287 per solved task. In the same runs, LiteLLM’s heuristic auto-router solved 72.8 percent on RouterArena and 40 of 77 coding tasks, according to the post. These are a vendor’s own numbers on default settings. Broader gateway cost controls are covered in LLM cost control with an AI gateway.

Upgrade notes for operators

The v2.3.0 upgrade notes list two decision-related actions:

• Map-form bodies. Map-form /v1/decisions requests are deprecated but still served. Go SDK users should move to the new BifrostDecisionRequest fields.
• Emulation. Decisions sent to non-decision models now return unsupported_operation. Deployments that relied on chat-model fallbacks for decisions should either set compat.convert_decision_to_responses: true, send x-bf-compat: ["convert_decision_to_responses"], or change fallback chains to name native decision models.

The release adds a compat_convert_decision_to_responses column (default false) and decision pricing columns to the config store. v2.3.1 admits custom OpenAI-format providers to /v1/decisions by their allowed request types alone.

Open questions

Calibration across providers. A single endpoint makes it easy to swap Jev for Clef or a Perplexity decider by changing one string. It does not tell an application whether a 0.8 from one model means the same as a 0.8 from another. The announcement advises evaluating “a new model against representative inputs” first. The same applies to fallbacks.

Dialect drift. OpenAI’s format is adding prompt caching and per-question images; TypeSafe’s has the experimental box question in Jev 1.14. A normalising layer has to track both.

Unpriced providers. Azure and Databricks decisions record $0 until an operator adds an override, and Databricks returns no usage figures for decisions. Budget enforcement on those paths depends on configuration, not defaults.

Provider maturity. Databricks labels ai_decide Beta. OpenRouter’s path is /api/alpha/decisions. Ollama calls tev1 experimental. A unified API makes these easy to reach; it does not change their status.

Decision models arrived as a single vendor’s product in September. Six weeks later they have a second dialect from the largest model provider, input-only pricing across the board, and local runtimes. A gateway that treats decisions as a separate request type, with its own endpoint, price fields and routing condition, is betting that this traffic will be large enough to manage on its own terms. Turning emulation off by default supports that bet. It keeps the category’s main claim, a probability that means something, from being quietly replaced by a chat model’s guess at full chat prices.