Budgets & Limits

Budget AI enforces two kinds of control before any provider is called:

• Rate limits cap requests and tokens per minute, per key and per project.
• Budgets cap USD spend per day, week or month, per key, tag, org unit and project.

A request is checked against every limit and every budget that applies to it, in one step that is shared by all gateway replicas. A refused request never reaches the provider and consumes nothing.

Rate limits

An empty limit means no limit. Limits accept 1 to 10,000,000.

How they are counted:

• Windows are fixed calendar minutes, timed by the shared Redis clock so every replica agrees.
• Each request counts 1 towards RPM.
• For TPM, a request counts its estimated input tokens (about one token per four bytes of messages and tools) plus its output allowance: max_tokens or max_completion_tokens if the caller set one, otherwise 4096. When the window has less room than that, the output allowance shrinks to fit; a request whose input alone exceeds the room left is refused.
• After the response, the difference between the estimate and the provider's reported usage is corrected in the same window, if it has not rolled over.

A refused request receives 429 rate_limit_exceeded (RPM) or tokens_limit_exceeded (TPM) with a retry-after header giving the seconds left in the minute.

Budgets

A budget is a USD amount for a period, attached to one target. Budgets are managed in Budget AI → Budgets & Limits and need gatewayAi.budgets:admin.

Periods are in UTC. Daily budgets reset at midnight UTC, weekly budgets on Monday (ISO weeks) and monthly budgets on the first of the month. A period resets by starting a fresh counter, so there is no reset job that could be missed.

Which budgets apply to a request

A request is limited by all of these at once, in this order:

1. The key's own budget.
2. One budget per tag the key carries.
3. The budget of the key's org unit, then each unit above it up to the top of the tree.
4. The project budget.

Any hard budget that would be exceeded refuses the request with 429 budget_exceeded. The message names the first exhausted budget in that order, for example The budget for org unit 'Engineering' is exhausted., because it is the one closest to the caller's control. Soft budgets never refuse; they record threshold crossings.

To see the chain for one key, open Budget AI → Keys, open the key's menu and select Limits. It lists every budget that applies with its period, amount, spend and remaining, and marks the hard budget with the least remaining as binding.

Reservations

A budget must hold even when many requests run at the same time. Budget AI therefore reserves each request's worst-case cost when it is admitted and settles the actual cost when it finishes.

• The reservation prices the estimated input tokens plus the output allowance (max_tokens or max_completion_tokens if set, otherwise 4096 tokens) at the provider model's price.
• A request is refused when spend so far, plus reservations already held, plus its own reservation would exceed a hard budget.
• When the request finishes, for any reason, the reservation is released and the real cost is added to spend. If a gateway replica stops before settling, the reservation expires after the model group's request timeout plus 30 seconds.

Reserving on the worst case means a caller close to the limit who asks for a large max_tokens can be refused even though the actual answer would have fitted. Setting max_tokens close to what you need keeps reservations small.

Threshold crossings

When spend passes one of a budget's Alert at percentages, the crossing is recorded once per threshold per period. Crossings appear in Threshold crossings on Budgets & Limits and in the Crossings count. Raising a budget's amount above current spend re-arms its thresholds.

Crossings are recorded in the product only. Sending them to Slack, PagerDuty or email is not available yet; see the Roadmap.

Enforced and recorded spend

The Budgets table shows two figures for each budget:

• Enforced spend is the real-time counter the gateway refuses against.
• Recorded spend is the durable figure written from that counter to the platform database every 10 seconds.

They should agree within 1 %. A warning appears when they do not.

Org units and the budget hierarchy

Org units group keys into departments, teams and sub-teams so a budget can be shared by every key inside a unit. Manage them on the Hierarchy tab of Budgets & Limits.

• Units form a tree up to four levels deep. The project budget sits above the top-level units.
• A key belongs to at most one unit. A key in no unit is independent: it is limited by its own budget, its tag budgets and the project budget.
• A key's chain can hold at most ten budgets.
• Create a unit, rename it, move it under another unit, and move keys between units from the Hierarchy tab. A unit with units or keys beneath it cannot be deleted.
• Moving a key between units, or issuing a key into a unit, needs gatewayAi.budgets:admin, because taking a key out of its unit escapes that unit's cap.

Three styles follow from the same tree:

If a key's chain cannot be built (for example its unit was removed, or the chain exceeds the limits above), every request with the key is refused with 503 budget_config_invalid rather than served without the unit's budget.

Allocation

A project or org unit budget has an allocation rule for the budgets beneath it, set in Budgets inside it:

• Each can be up to this amount (default): each budget below may be as large as this one; together they may add up to more.
• Must fit within this amount together: the nearest budgets below must add up to no more than this amount. A unit with no budget of its own draws on whatever the others leave unallocated.

Every budget is also checked against every budget above it. Budgets with different periods are compared as a daily rate, using a 30-day month and a 7-day week. A change is refused only for a violation it introduces, so an older inconsistency never blocks an unrelated edit. Tag budgets sit outside the tree and are not checked against it.

Unit spend

Every request adds its cost to each unit in the key's chain, whether or not the unit has a budget. Unit spend is counted per calendar month, whatever period the unit's own budget uses. Moving a key or a unit does not move spend already recorded; new spend goes to the new parent.

Hierarchy insights

The Hierarchy tab shows each unit's spend, keys and budgets, the units beneath it, where the spend goes, and how much of the project budget is committed to budgets below it. For budgets part-way through a period it projects the spend to the end of the period and, when the current rate would pass the limit, roughly when that will happen.

The Budgets & Limits page

The Budgets & limits tab summarises the selected time range: spend, the project budget, budgets set, budgets at or above 80 %, requests refused by budget, threshold crossings this period, and spend over time by model group. Its Budget pressure card shows every budget against how far through its period it is, so a budget spending faster than the clock stands out. Spend from the platform-managed Playground key and from deleted keys is grouped together in the charts.

Organization view

Organization Settings → Budget AI lists every project in the organization with Budget AI enabled, with requests, cost, error rate and spend against budget over the last 24 hours. It needs gatewayAi.budgets:read at organization level.

Refusal summary

Refused requests are traced with zero usage and gateway.refusal set to the code, and are counted on the Overview Refusals card. See Observability & Cost.