Token budget
The token allowance you choose for a request or session. It caps cost and size below the model's absolute context-window limit.
See it
What it is
A token budget is the allowance you set for how many input, output, and sometimes hidden reasoning tokens a request or session may consume. It is an application rule, not the model's context window. The window says what can fit; the budget says what you are willing to send, wait for, and pay for.
Set one before assembling the request. Reserve output room first, then divide the input allowance among system instructions, retrieved evidence, summaries, and recent messages in priority order. Per-request budgets prevent failures, while per-user or per-session budgets keep a long workflow from quietly spending without limit.
Gotcha: an output cap is not a total budget. A short answer can still follow a huge input, and a reasoning model can spend billed tokens that never appear in the visible reply. Count the serialized request with the selected model's tokenizer and reconcile estimates against the usage object returned by the API.
Ask AI for it
Implement token-budget middleware for the OpenAI Responses API. Count the fully serialized input with tiktoken, reserve max_output_tokens plus a 10 percent safety margin, and enforce inputBudget = contextLimit - outputReserve - margin before every call. Keep system instructions and the newest user message, summarize older turns, then drop the lowest-ranked retrieved chunks until the request fits. Reject any request that still exceeds the budget, and log estimated tokens beside response.usage.input_tokens and response.usage.output_tokens so counting drift is visible.