API Reference
Budget AI serves an OpenAI-compatible API at https://<your-gateway-host>/v1. Point any OpenAI SDK at that base URL and use a Budget AI key as the API key. The exact address for your deployment is shown in Project Settings → Budget AI → Budget AI address.
This page covers the gateway's own API and the key minting API for short-lived agent keys. Managing model groups and budgets, and issuing other keys, is done in the product; those operations are not part of the public REST API.
Authentication
Send the key in either header:
Requests must use Content-Type: application/json. The default body limit is 4,718,592 bytes (4.5 MiB).
Routes
| Method | Path | Purpose |
|---|---|---|
POST | /v1/chat/completions | Chat completions, streaming or not |
POST | /v1/embeddings | Embeddings |
GET | /v1/models | The model groups this key may call |
Any other path returns 404 with code unknown_route.
POST /v1/chat/completions
The body follows the OpenAI Chat Completions API.
| Field | Required | Notes |
|---|---|---|
model | No | A model group name. May be omitted or empty when the key is granted exactly one model group. |
messages | Yes | A non-empty array; each message needs a role |
stream | No | true returns server-sent events |
stream_options | No | As in the OpenAI API |
tools, temperature, max_tokens, max_completion_tokens, response_format and other parameters | No | Sent on to the provider. For Anthropic, AWS Bedrock and Google Vertex AI the request is translated to the provider's API. |
The response is an OpenAI chat completion object. For a non-OpenAI provider it is translated to the same shape, including usage.
Streaming
With "stream": true the response is text/event-stream in the OpenAI chunk format, ending with data: [DONE]. Headers are sent when the provider sends its first byte; until then Budget AI can still retry or fail over. If the provider stream breaks after that, the stream ends with an error event and [DONE]:
For OpenAI-compatible and Azure OpenAI providers, Budget AI asks for a final usage chunk so streamed calls are costed from real usage.
When AI Guardrails inspection applies to the key, a stream can also end with an error event coded guardrail_response_blocked or guardrail_unavailable, and can carry SSE comments: : keepalive while the gateway waits for inspection, and : x-ants-guardrail-warning: <categories> just before [DONE] for a Warn result. OpenAI SDKs ignore comments. See What the caller sees.
max_tokens matters for streaming and non-streaming calls alike: it sets the output allowance used for rate limits and budget reservations. Without it, the model group's Default output reserve is assumed (4,096 tokens when it is not set). Under a hard token budget, Budget AI can send the request with a lower output limit than you asked for and say so in the x-ants-max-tokens-capped header; see Output cap.
POST /v1/embeddings
The body follows the OpenAI Embeddings API.
| Field | Required | Notes |
|---|---|---|
model | No | A model group name, with the same single-group rule as chat |
input | Yes | Text or an array, as in the OpenAI API |
Embeddings are served only by deployments whose provider family has an embeddings API, which today is the OpenAI-compatible family. Streaming does not apply. The response cache does not apply. Usage is priced on input tokens.
GET /v1/models
Lists the model groups the key is granted, sorted by name:
id is the model group name to send as model. created is when the group was created.
Caller context headers
langfuse-user-id, langfuse-session-id, langfuse-tags, langfuse-metadata and traceparent are accepted on chat completions and embeddings to attribute the request in Tracing. See Caller context headers.
When the model is served through a Portkey upstream, x-portkey-metadata (a flat JSON object, up to 4,096 characters) and x-portkey-trace-id (up to 128 characters) are passed on to Portkey on chat completions. x-portkey-api-key and x-portkey-config are never passed from the caller. See Headers passed through to Portkey.
Response headers
| Header | When | Value |
|---|---|---|
x-ants-request-id | Every chat completions and embeddings response, including errors | The request id, which is also the trace id in Tracing |
x-ants-model-group | When the model group was found | The group that served the request, which is the fallback group when one was used |
x-ants-deployment | When a deployment was chosen | The deployment id |
x-ants-cost | Served requests | Cost in USD. For streams, the estimate made at admission. |
x-ants-cost-estimated | Served requests | true for streams, otherwise false |
x-ants-attempts | When at least one provider attempt was made | JSON list of attempts: d deployment id, o outcome (ok, retryable, failed, timeout, benched), ms duration, and where available st provider status and why provider message |
x-ants-cache | Response cache hits | hit |
x-ants-max-tokens-capped | Chat completions whose output limit a hard token budget lowered, see Output cap | The output limit the request was sent with (per completion when n is above 1) |
x-ants-guardrail-warning | AI Guardrails found content under a Warn action | Comma-separated category names: pii, pci, phi, ethicsBias, safety. Never content. On a stream, response warnings arrive as an SSE comment instead. |
retry-after | Rate limit refusals | Seconds until the current minute ends |
Errors
Errors use the OpenAI error envelope, so OpenAI SDKs raise their usual exception types:
type follows the status: authentication_error (401), permission_error (403), rate_limit_error (429), invalid_request_error (other 4xx) and server_error (5xx).
| Status | code | Cause | Traced |
|---|---|---|---|
| 400 | invalid_request | The body fails validation, for example messages is missing or empty | No |
| 400 | invalid_json | The body is not valid JSON | No |
| 400 | embeddings_not_supported | No deployment in the model group offers an embeddings API | Yes |
| 400 | guardrail_blocked | AI Guardrails blocked the request, or it carries tool calls outside the agent's declared tools. The provider is not called. The message is the policy's Blocked Response Message when set. | Yes |
| 400 | guardrail_response_blocked | AI Guardrails blocked the model's response, or the response calls tools outside the agent's declared tools. On a stream that has started, it arrives as an error event. | Yes |
| 400 | guardrail_unsupported_request | AI Guardrails applies to the key and the request sets n above 1 | Yes |
| 401 | invalid_api_key | The key is missing or unknown, or serving is off for its project | No |
| 401 | key_blocked | The key is blocked | Yes, sampled |
| 401 | agent_suspended | The key is bound to an agent that was suspended, see Suspend an agent. The message is "This agent has been suspended by an administrator." | Yes, sampled |
| 401 | key_expired | The key is past its expiry or its rotation grace window | Yes, sampled |
| 403 | ip_not_allowed | The request comes from outside the key's allowed source networks | Yes, sampled |
| 403 | model_not_allowed | The key is not granted the model group, the group does not exist, or model was omitted on a key with zero or several groups | Yes |
| 404 | unknown_route | The path is not served | No |
| 409 | budget_unpriced_model | A hard dollar budget applies and the provider model has no price | Yes |
| 413 | request_too_large | The body exceeds the size limit | No |
| 415 | unsupported_media_type | The body is not JSON | No |
| 429 | rate_limit_exceeded | A key or project requests-per-minute limit is reached | Yes |
| 429 | tokens_limit_exceeded | A key or project tokens-per-minute limit is reached | Yes |
| 429 | budget_exceeded | A hard budget would be exceeded, or a hard token budget has fewer output tokens left than the project's minimum capped output; the message names the budget | Yes |
| 502 | upstream_unreachable | No provider could be reached | Yes |
| 502 | upstream_invalid_response | A provider answered 2xx with a body that is not a JSON object | Yes |
| 503 | no_deployment_available | The model group has no eligible deployment | Yes |
| 503 | budget_config_invalid | The key's org unit budget chain cannot be built | Yes |
| 503 | guardrail_unavailable | AI Guardrails inspection applies, could not finish (time limit, detection service unavailable, or no text to inspect), and the project refuses what it cannot inspect. On a stream that has started, it arrives as an error event. | Yes |
| 503 | gateway_config_stale | The gateway cannot confirm its configuration is current, or its Redis is unavailable. Retry shortly. | No |
| 504 | upstream_timeout | No provider answered within the timeout | Yes |
| Provider's status | Provider's code, or upstream_error | The provider returned an error; its own error envelope is passed through when it has one | Yes |
| 200, then an error event | upstream_stream_error | The provider stream broke after it started | Yes |
| 500 | none | Unexpected gateway error | No |
When every deployment and fallback fails, the caller receives the last provider error (or upstream_unreachable / upstream_timeout), and the trace records gateway.refusal = all_deployments_failed.
"Sampled" means the refusal is traced at most once per minute per key and reason. "Traced" refusals carry zero usage. See Observability & Cost.
Retry 429 responses after the retry-after interval, and 503 gateway_config_stale after a short pause. Budget AI already retries provider failures inside the request according to the model group's routing policy, so aggressive client retries on 502 and 504 mostly add load.
Operational endpoints
These are for load balancers and monitoring, not for applications. They need no key.
| Endpoint | Port | Meaning |
|---|---|---|
GET /health | API port and admin port | 200 when the process is up and Redis answers |
GET /ready | API port and admin port | As /health, and 503 while the gateway is draining for shutdown or its configuration is stale |
GET /metrics | Admin port only | JSON counters for requests, refusals and errors by code, latency percentiles, telemetry, configuration and process health |
See Self-Hosting & Configuration for ports.
Key minting API
An authorised system can mint a short-lived Budget AI key for one agent. This endpoint is served by ANTS Platform, not by the gateway.
- Off by default. Your ANTS deployment turns it on. Until then every call returns 403.
- Authentication. HTTP Basic with an organization API key. A project API key is refused with 403, so an SDK key cannot mint Budget AI keys. The organization needs the API and Budget AI entitlements, and Budget AI must be enabled for the project.
| Request field | Required | Description |
|---|---|---|
agentId | Yes | The agent the key is bound to. It must exist in the project. |
modelGroupIds | Yes | IDs of the model groups the key may call, at least one |
expiresInMinutes | Yes | Lifetime in minutes, at least 1. It cannot exceed the project's maximum key expiry. |
budgetUsd | Yes | A key budget in USD. Like Monthly budget (USD, optional) on the issue form, it resets monthly and is alert only: it records threshold crossings but does not stop requests. See Short-lived keys for agents. Because it is in dollars, every mint in a project that budgets in tokens is refused with 400. |
alias | No | A name for the key, unique in the project. A name is generated when omitted. |
rpmLimit, tpmLimit | No | Requests and tokens per minute limits for the key |
tags | No | Tags for the key |
allowedCidrs | No | IPv4 or IPv6 addresses or CIDR ranges the key may be used from |
orgUnitId | No | Place the key in an org unit, so that unit's budget applies |
Fields that are not listed are refused.
A successful request returns 201 with the key. The response is sent with Cache-Control: no-store.
The plaintext key appears only in this response. Store it where the agent can read it, and do not log it. Every minted key is written to the audit log.
When the key can be used
A new key reaches the gateways within about 2 seconds. availableInMs is how long to wait before the first call; it is 0 when the key is already live. If propagated is false, the gateways pick the key up on their next full reload, up to 60 seconds later. Use a lifetime of at least 5 minutes.
Errors
| Status | Meaning |
|---|---|
| 400 | Invalid body, a lifetime above the project maximum, or the project budgets in tokens (This project budgets in tokens.) |
| 401 | Bad credentials |
| 403 | The API is off, the key is not an organization key, or the plan or Budget AI does not allow it |
| 404 | Project, agent or model group not found |
| 409 | The alias is already used in the project |
| 429 | Too many keys. A project can mint 60 keys per minute and an agent can hold 100 active keys. The response carries Retry-After: 60. |