API Reference
Budget AI serves an OpenAI-compatible API at https://<your-gateway-host>/v1. Point any OpenAI SDK at that base URL and use a Budget AI key as the API key. The exact address for your deployment is shown in Project Settings → Budget AI → Budget AI address.
This page covers the gateway's own API and the key minting API for short-lived agent keys. Managing model groups and budgets, and issuing other keys, is done in the product; those operations are not part of the public REST API.
Authentication
Send the key in either header:
Requests must use Content-Type: application/json. The default body limit is 4,718,592 bytes (4.5 MiB).
Routes
| Method | Path | Purpose |
|---|---|---|
POST | /v1/chat/completions | Chat completions, streaming or not |
POST | /v1/embeddings | Embeddings |
GET | /v1/models | The model groups this key may call |
Any other path returns 404 with code unknown_route.
POST /v1/chat/completions
The body follows the OpenAI Chat Completions API.
| Field | Required | Notes |
|---|---|---|
model | No | A model group name. May be omitted or empty when the key is granted exactly one model group. |
messages | Yes | A non-empty array; each message needs a role |
stream | No | true returns server-sent events |
stream_options | No | As in the OpenAI API |
tools, temperature, max_tokens, max_completion_tokens, response_format and other parameters | No | Sent on to the provider. For Anthropic, AWS Bedrock and Google Vertex AI the request is translated to the provider's API. |
The response is an OpenAI chat completion object. For a non-OpenAI provider it is translated to the same shape, including usage.
Streaming
With "stream": true the response is text/event-stream in the OpenAI chunk format, ending with data: [DONE]. Headers are sent when the provider sends its first byte; until then Budget AI can still retry or fail over. If the provider stream breaks after that, the stream ends with an error event and [DONE]:
For OpenAI-compatible and Azure OpenAI providers, Budget AI asks for a final usage chunk so streamed calls are costed from real usage.
max_tokens matters for streaming and non-streaming calls alike: it sets the output allowance used for rate limits and budget reservations. Without it, 4096 output tokens are assumed.
POST /v1/embeddings
The body follows the OpenAI Embeddings API.
| Field | Required | Notes |
|---|---|---|
model | No | A model group name, with the same single-group rule as chat |
input | Yes | Text or an array, as in the OpenAI API |
Embeddings are served only by deployments whose provider family has an embeddings API, which today is the OpenAI-compatible family. Streaming does not apply. The response cache does not apply. Usage is priced on input tokens.
GET /v1/models
Lists the model groups the key is granted, sorted by name:
id is the model group name to send as model. created is when the group was created.
Caller context headers
langfuse-user-id, langfuse-session-id, langfuse-tags, langfuse-metadata and traceparent are accepted on chat completions and embeddings to attribute the request in Tracing. See Caller context headers.
Response headers
| Header | When | Value |
|---|---|---|
x-ants-request-id | Every chat completions and embeddings response, including errors | The request id, which is also the trace id in Tracing |
x-ants-model-group | When the model group was found | The group that served the request, which is the fallback group when one was used |
x-ants-deployment | When a deployment was chosen | The deployment id |
x-ants-cost | Served requests | Cost in USD. For streams, the estimate made at admission. |
x-ants-cost-estimated | Served requests | true for streams, otherwise false |
x-ants-attempts | When at least one provider attempt was made | JSON list of attempts: d deployment id, o outcome (ok, retryable, failed, timeout, benched), ms duration, and where available st provider status and why provider message |
x-ants-cache | Response cache hits | hit |
retry-after | Rate limit refusals | Seconds until the current minute ends |
Errors
Errors use the OpenAI error envelope, so OpenAI SDKs raise their usual exception types:
type follows the status: authentication_error (401), permission_error (403), rate_limit_error (429), invalid_request_error (other 4xx) and server_error (5xx).
| Status | code | Cause | Traced |
|---|---|---|---|
| 400 | invalid_request | The body fails validation, for example messages is missing or empty | No |
| 400 | invalid_json | The body is not valid JSON | No |
| 400 | embeddings_not_supported | No deployment in the model group offers an embeddings API | Yes |
| 401 | invalid_api_key | The key is missing or unknown, or serving is off for its project | No |
| 400 | guardrail_blocked | Guardrail checks found a violation in the prompt, or in a non-streamed response, and the policy blocks it. The message is the policy's Blocked Response Message. | Yes |
| 401 | key_blocked | The key is blocked, including when its agent is suspended | Yes, sampled |
| 401 | key_expired | The key is past its expiry or its rotation grace window | Yes, sampled |
| 403 | ip_not_allowed | The request comes from outside the key's allowed source networks | Yes, sampled |
| 403 | model_not_allowed | The key is not granted the model group, the group does not exist, or model was omitted on a key with zero or several groups | Yes |
| 404 | unknown_route | The path is not served | No |
| 409 | budget_unpriced_model | A hard budget applies and the provider model has no price | Yes |
| 413 | request_too_large | The body exceeds the size limit | No |
| 415 | unsupported_media_type | The body is not JSON | No |
| 429 | rate_limit_exceeded | A key or project requests-per-minute limit is reached | Yes |
| 429 | tokens_limit_exceeded | A key or project tokens-per-minute limit is reached | Yes |
| 429 | budget_exceeded | A hard budget would be exceeded; the message names the budget | Yes |
| 502 | upstream_unreachable | No provider could be reached | Yes |
| 502 | upstream_invalid_response | A provider answered 2xx with a body that is not a JSON object | Yes |
| 503 | no_deployment_available | The model group has no eligible deployment | Yes |
| 503 | budget_config_invalid | The key's org unit budget chain cannot be built | Yes |
| 503 | guardrail_unavailable | Guardrail checks are on, the detection service cannot be reached, and the policy is set to fail closed | Yes |
| 503 | gateway_config_stale | The gateway cannot confirm its configuration is current, or its Redis is unavailable. Retry shortly. | No |
| 504 | upstream_timeout | No provider answered within the timeout | Yes |
| Provider's status | Provider's code, or upstream_error | The provider returned an error; its own error envelope is passed through when it has one | Yes |
| 200, then an error event | upstream_stream_error | The provider stream broke after it started | Yes |
| 500 | none | Unexpected gateway error | No |
When every deployment and fallback fails, the caller receives the last provider error (or upstream_unreachable / upstream_timeout), and the trace records gateway.refusal = all_deployments_failed.
"Sampled" means the refusal is traced at most once per minute per key and reason. "Traced" refusals carry zero usage. See Observability & Cost.
Retry 429 responses after the retry-after interval, and 503 gateway_config_stale after a short pause. Budget AI already retries provider failures inside the request according to the model group's routing policy, so aggressive client retries on 502 and 504 mostly add load.
Operational endpoints
These are for load balancers and monitoring, not for applications. They need no key.
| Endpoint | Port | Meaning |
|---|---|---|
GET /health | API port and admin port | 200 when the process is up and Redis answers |
GET /ready | API port and admin port | As /health, and 503 while the gateway is draining for shutdown or its configuration is stale |
GET /metrics | Admin port only | JSON counters for requests, refusals and errors by code, latency percentiles, telemetry, configuration and process health |
See Self-Hosting & Configuration for ports.
Key minting API
An authorised system can mint a short-lived Budget AI key for one agent. This endpoint is served by ANTS Platform, not by the gateway, and uses the same project API keys (HTTP Basic auth) as the other public APIs.
| Request field | Required | Description |
|---|---|---|
agentId | Yes | The agent the key is bound to |
modelGroups | Yes | Names of the model groups the key may call, at least one |
expiresInHours | Yes | Lifetime in hours. It cannot exceed the project's maximum key expiry. |
alias | No | A name for the key, unique in the project. A name is generated when omitted. |
budgetUsd | No | A key budget in USD that stops requests when reached |
rpm, tpm | No | Requests and tokens per minute limits for the key |
allowedNetworks | No | IPv4 or IPv6 addresses or CIDR ranges the key may be used from |
A successful request returns 201 with the key:
The plaintext key appears only in this response. Store it where the agent can read it, and do not log it. A request with an unknown agent or model group, or a lifetime above the project maximum, is refused with 400 and a message naming the field. Every minted key is written to the audit log.