The Shift Is Not the Discount, but the Operating Surface
On September 22, 2026, OpenAI announced an upgrade to prompt caching for GPT-6, aimed at agents that run for hours and make a sequence of API calls to complete complex tasks. These applications repeatedly carry forward system instructions, tool definitions, and earlier context. OpenAI reuses those shared prefixes to avoid repeated computation, while offering discounts of up to 90% on eligible cached input tokens reused within a 30-minute window.
The discount is only the visible part of the change. For long-running agents, cache hit rate also affects time to first response, cost per task, and the ability to scale reliably. The new dashboard, miss diagnostics, explicit breakpoints, and prewarming controls show that caching is no longer merely an invisible optimization inside the model service. It is becoming an operational layer that application teams must observe and govern.
An Agent’s Cache Is a Context That Keeps Changing
The system does not cache an isolated prompt. It reuses a stable prefix across a sequence of requests. Instructions, tool definitions, schemas, tool ordering, and accumulated reference material can all remain reusable when they stay consistent. But a change to tool definitions can invalidate the reusable prefix, even when the change appears limited to one part of the context.
OpenAI’s diagnostic example turns that behavior into an actionable failure. A request missed the cache because tools_changed was reported, and the comparison estimated that 5,629 reusable tokens had also become missed tokens. The value of this number is not its size alone. It tells the team what changed and lets them distinguish a model, tool, setting, or input change from a mysterious drop in cache performance.
The New Tools Put Caching into the SRE Workflow
The Prompt Caching Dashboard shows hit rates over time and the composition of cached versus uncached input tokens. Teams can compare application changes with caching behavior and determine whether a tool update, context restructure, or request-policy change caused a decline. The diagnostic tool explains individual misses and estimates the affected token volume, making caching something that can be investigated alongside latency and error rate.
Developers can also use explicit cache breakpoints to choose which prefixes should be reused, and prewarm shared instructions, tool definitions, or reference material before a user sends a request. GPT-6 additionally allows reasoning effort to change between responses without breaking the existing cache. A system can therefore spend more reasoning on a difficult task and less on a routine follow-up while preserving previously processed context. Caching becomes a matter of boundary design, capacity analysis, and runtime control rather than a hope that reuse will happen.
The Cost Is That Tool Flexibility Must Yield to Stability
To preserve reuse, an application needs stable tool definitions, schemas, and ordering. OpenAI recommends keeping definitions in place and using allowed_tools to restrict which tools are callable, or setting tool_choice to none when no tool is needed, instead of removing definitions. New developer messages can also be appended near the end of the context to override older instructions without rewriting the shared prefix.
This makes tool version governance part of the agent architecture. It can reduce cache invalidation, but an interface change must now be evaluated not only for functional correctness, but also for whether it destroys reuse across a large context. Explicit breakpoints, prewarming, and diagnostics also require maintenance. A discount of up to 90% on cached input tokens is not the same as a 90% reduction in end-to-end cost, because tool calls, misses, context maintenance, and operational work remain.
The Metric That Matters Is Cost per Task, Not per Request
The deployment feedback in the material shows that the change can produce meaningful operational results. GitHub Copilot reported that, across billions of requests, the share of prompt tokens requiring fresh processing fell by more than 50% from its previous baseline. In a production deployment, Manus reported that after adjusting cache breakpoints, combining explicit and automatic caching, and using real requests to investigate abnormal misses, its hit rate rose from roughly 85% to consistently above 90% in less than a week.
These figures should not be treated as universal outcomes, since workloads, context length, and tool-change frequency vary. They do suggest a more useful operating model: long-running agents should track cache hit rate, missed tokens, time to first response, and cost per task together. For a technical leader, the goal is not to maximize hit rate at any price. The goal is to find a context structure that preserves task economics without making tool evolution unnecessarily slow.