Evidence at a glance
Small Decisions Can Add Up to Workflow Delay
On October 22, 2026, OpenAI announced the general availability of the Decisions API. It is designed for small judgments inside application and agent workflows, such as classifying an input, routing a request, or choosing the next action. It is not a new entry point for open-ended conversation. Instead, developers define a question and its allowed answers, then provide text, images, or structured input for the model to evaluate.
The design targets not the latency of one isolated call, but the cost that accumulates when judgments recur throughout a workflow. A request may pass through classification, routing, and action selection. Each wait may be brief, yet together they can slow an interactive or automated path and add to the cost of serving it. The product tradeoff is to replace some general model calls with more focused decision calls, prioritizing steps whose answers are limited but whose frequency is high.
Bounded Answers Give Software Something to Act On
To use the API, developers define the question and the allowed answers, then provide the relevant context. The Decisions API supports three answer types: a probability from 0 to 1 estimating whether a condition is true, a choice among predefined options, or a score. An application can use the result to choose what happens next without first parsing open-ended prose into an instruction the software can execute. Text, images, and native JSON inputs allow the judgment to be applied to different kinds of business data.
That does not make a model output a business rule by itself. A probability estimates how likely a condition is to hold, while a choice or score is also a model judgment. The application still needs to define thresholds, permissions, and exception handling. OpenAI says the Decisions API is built with GPT-6 Luna and can be up to 10 times faster on bounded questions than calling GPT-6 Luna through the Responses API. That figure is a vendor-stated upper bound, not a multiplier guaranteed across every task, input size, or deployment condition.
New Capabilities Address Different Integration Constraints
At general availability, OpenAI announced several distinct updates: native GPT-Live integration, prompt caching for reused context, inference residency in the US and EU, and native JSON input. A request can now contain up to 400 questions, twice the previous limit. OpenAI also says it has added capacity in several Asian regions and continued to improve latency. These changes are not one undifferentiated performance upgrade. They address different needs, including live interaction, repeated context, structured data, batched judgments, and deployment location.
The updates expand the available integration paths, but they do not shift responsibility away from the application. GPT-Live integration is relevant to real-time voice scenarios, prompt caching addresses reused context, and the higher question limit creates more room for related judgments in one request. Residency information may matter to teams with regional processing requirements, but the material does not establish that it satisfies every organization's specific compliance obligations. Teams still need to check their own data flows, regional requirements, and response targets rather than treating one API feature as a complete deployment plan.
Customer Results Depend on Where the Decision Sits
The published cases show that the API's value depends heavily on where the judgment sits. In an evaluation of 30 certificate-of-insurance cases, Box says route selection was about five times faster, with the same accuracy as the Responses API. Lower-confidence cases still went to human reviewers. Datadog placed the API before an automated remediation action to score risks in agent-generated rshell commands. It reported 40 times faster p95 latency than available frontier models, with comparable accuracy. One case speeds document routing, while the other adds a fast judgment before autonomous execution. Neither removes human or system controls from the process.
Other cases cover cost attribution, customer conversations, and language learning. Ramp reports that calls for attributing AI costs were four times faster and 30% cheaper, without sacrificing accuracy. Sierra says calls within parts of its customer-conversation orchestration run below 280 milliseconds p95, about three times faster and 90% cheaper per call, with no drop in customer outcomes. Speak's testing cut time to the next action in its GPT-Live integration from 1.4 seconds to 0.13 seconds, a reduction of more than 90%. These figures come from different teams, baselines, and tasks. They do not form a single ranking and cannot be directly projected onto products with different input distributions.
Evaluate the Whole Workflow, Not Just the Call
For technical leaders, the key evaluation question is not how much faster an isolated call can be. It is whether the API improves the end-to-end workflow without introducing unacceptable errors. Start with a frequent task whose answer boundaries are clear and whose result can be checked, then compare latency, cost, and accuracy against the existing Responses API path. If the task is rare, or if the cost of an error means every result must be reviewed by a person, shortening one call may not produce a meaningful system-level gain.
More importantly, uncertainty has to be designed into the execution path. For outputs expressed as scores or probabilities, teams need to decide when to automate, when to fall back, and who reviews low-confidence cases. When a model result can trigger command execution or another consequential action, permission checks, handling for unusual inputs, and human intervention cannot be replaced by speed. The Decisions API offers a focused route for a judgment. Reliability still depends on the boundaries of the question, the evaluation method, and how the application handles the result.