OpenAI has refreshed GPT-6 prompt caching to increase reuse of shared context across requests, an upgrade aimed at cutting response latency and token costs for applications that repeatedly send the same instructions, tools or reference material. The company says the new system raises cache hit rates by default, extends a 30-minute shared-prefix discount window and introduces diagnostics and controls so developers can monitor and tune caching behavior.
Persistent agents and long-running interactions are the primary beneficiaries of the update. Applications that break complex tasks into many linked requests—refactoring codebases, drafting long documents or producing multi-step analyses—often recompute the same prompt context on each call. To avoid that redundant work, OpenAI caches shared prompt prefixes so later requests can reuse earlier computation. With the GPT-6 prompt caching changes, shared prefixes reused within a 30-minute window are eligible for discounted cached input tokens, and OpenAI reports discounts of up to 90% for cached tokens in eligible cases.
To help teams measure reuse and spot problems, OpenAI added a Prompt Caching Dashboard. The dashboard shows hit-rate trends and an input composition chart that compares cached and uncached tokens, giving developers a clear view of how much input is being served from cache over time. This visibility is intended to make it easier to detect drops in reuse—whether from code changes, altered tool definitions or different request patterns—and to assess how integrations affect caching performance.
For unexpected misses, OpenAI now provides a prompt caching diagnostics tool that compares a request with a recent response to identify what prevented reuse. Diagnostic output flags whether a request was a cache miss, outlines likely causes (for example, changes to tools) and estimates how many tokens were affected. The diagnostics help teams prioritize fixes and estimate the impact of misses on both latency and cost.
The update also introduces developer controls to tune caching without sacrificing flexibility. Explicit cache breakpoints let developers mark which prompt prefixes should be reused and which should not, while documentation clarifies how long cached prefixes remain eligible and how changes to tools or input affect reuse. On GPT-6 models, teams can change internal reasoning effort between responses without breaking cache: by appending a configuration_update while keeping the request-level reasoning effort unchanged, developers can increase or decrease computational effort for a particular task while preserving reusable context.
To maintain cached context as tool usage evolves, OpenAI recommends keeping tool definitions, schemas and ordering stable. Rather than removing tool definitions—which can invalidate cached prefixes—applications can restrict callable tools via allowed_tools or set tool_choice to none when tools are not needed. New developer messages permit adding instructions at the end of the context so newer guidance overrides earlier text without invalidating the cached prefix. These tactics aim to reduce cache churn as applications iterate on toolsets and instructions.
Another practical addition is prewarming: preparing common context ahead of user requests so the model can begin responding more quickly when a request arrives. Prewarming shifts processing out of the user’s wait time by loading shared instructions, tool definitions or reference material into cache before they are needed, increasing the chance of immediate reuse when traffic arrives.
OpenAI frames the changes as directly affecting both performance and operating costs. By reusing computation across requests, the improved prompt caching reduces response latency for repeated context and provides discounts on cached input tokens; the company cites savings of up to 90% for cached tokens in some cases. For applications that maintain long interactions or run persistent agents for hours, higher cache hit rates combined with diagnostic tooling and explicit controls can materially lower waiting times and token expenses.
To get started, OpenAI recommends monitoring the Prompt Caching Dashboard, investigating unexpected misses with the diagnostics tool and following the prompt caching guide to refine integrations. The guide also suggests using Codex to review implementation code, apply improvements and measure results. Links to the dashboard, diagnostics, configuration guidance, prewarming instructions and other documentation are available from OpenAI’s prompt caching documentation.
Taken together, the enhancements give developers more observability and control over when and how context is reused in multi-request workflows. For teams running persistent agents or otherwise sending repeated context to GPT-6, the changes offer a route to faster responses and lower token bills without sacrificing the ability to tune model behavior.
Source: Read the original source
Leave a Reply