Budget AI
Budget AI is the AI gateway built into ANTS Platform. Your applications and agents call a single OpenAI-compatible endpoint with an ANTS-issued key. Budget AI checks the key, applies rate limits and budgets, picks a provider deployment, retries or fails over when a provider has trouble, and records the request as a trace in your project with its cost.
What it does
| Capability | What you get |
|---|---|
| One endpoint | https://<your-gateway-host>/v1 speaks the OpenAI API: chat completions (streaming and non-streaming), embeddings and a model list. Any OpenAI SDK works by changing the base URL and key. |
| Many providers | OpenAI and any OpenAI-compatible API, Azure OpenAI, Anthropic, AWS Bedrock and Google Vertex AI (including a Gemini API key). See Providers & Credentials. |
| Model groups and routing | Applications call a name you choose, such as prod-chat. Behind it you place one or more provider models, a routing strategy, retries, cooling-off for failing deployments and fallback groups. See Model Groups & Routing. |
| Keys | Keys that expire after hours or days, are limited to named model groups, can be confined to source networks, can be bound to an agent, and can be rotated, blocked and deleted. Short-lived keys for an agent can be minted by API, and suspending an agent blocks all its keys at once. See Keys. |
| Guardrails | Optional inline checks of prompts and non-streamed responses with the same policies as AI Guardrails, switched on per project or per key. See Guardrails. |
| Limits and budgets | Requests and tokens per minute per key and per project, and daily, weekly or monthly USD budgets per key, tag, org unit and project, with hard stops or alerts. See Budgets & Limits. |
| Tracing and cost | Every request that reaches a known key becomes a trace with cost, usage, latency and routing detail, and can carry user, session, tags and metadata. See Observability & Cost. |
How it fits with the rest of ANTS Platform
Budget AI writes its traces through the same ingestion path as the SDKs and OpenTelemetry. Gateway traffic therefore appears in Tracing, Sessions, Users, dashboards and cost reporting next to the rest of your AI traffic, priced with your project's model prices. Each gateway trace carries the gateway tag and a Gateway panel that shows the key, model group, deployment and attempts.
Spend control happens before the provider is called. A request that would exceed a hard budget or a rate limit is refused and never reaches the provider.
Where to find it
| Place | What is there |
|---|---|
| Budget AI → Overview | Traffic, spend, errors, refusals, budgets, busiest keys and model health for the project |
| Budget AI → Keys | Issue, rotate, block, unblock and delete keys; per-key traffic and spend |
| Budget AI → Budgets & Limits | Budgets, spend, threshold crossings and the org unit Hierarchy |
| Budget AI → Models & Routing | Provider credentials, model groups, deployments, routing, fallbacks and cache |
| Project Settings → Budget AI | Turn serving on or off, the gateway address and SDK snippets, key expiry policy, project rate limits, guardrail checks, provider secret storage and health checks |
| Organization Settings → Budget AI | Every project with Budget AI enabled, with requests, cost, error rate and spend against budget over the last 24 hours |
What you need to turn it on
Budget AI appears only when all of these hold:
- Plan entitlement. The organization's plan includes the
llm-gatewayentitlement. It is part of the Enterprise plan on ANTS Cloud and of an Enterprise self-hosted license. - Feature flag. The
gatewayAifeature flag is enabled for the user. Instance administrators, and deployments that setLANGFUSE_ENABLE_EXPERIMENTAL_FEATURES=true, see it without the flag. Contact your account team to have it enabled. - Product module. The
gatewayproduct module is not hidden for the deployment (for example throughLANGFUSE_UI_HIDDEN_PRODUCT_MODULES). - The gateway service. On ANTS Cloud it is operated for you. On a self-hosted deployment, enable the Helm subchart with
gateway.enabled: trueand setglobal.gateway.publicBaseUrl. See Self-Hosting & Configuration. - Permissions. The user holds the Budget AI permissions for what they want to do. See Access control.
- Per project. An administrator turns on Serve gateway traffic for this project in Project Settings → Budget AI. It is off for a new project.
The checks in steps 1 to 3 are enforced on the server for every Budget AI action, not only in the navigation. The gateway itself serves a project only while serving is on and the organization holds the entitlement.
Budget AI screens carry no beta label. Access is still granted per user through the gatewayAi feature flag.
Access control
Budget AI permissions are part of the Budget AI module in roles and custom roles. Each page has its own permission scope:
| Scope | Allows |
|---|---|
gatewayAi.overview:read | View the Overview page |
gatewayAi.keys:read | List keys and outstanding collection links, see key traffic |
gatewayAi.keys:write | Issue keys and collection links, rotate keys, change a key's allowed networks |
gatewayAi.keys:admin | Block, unblock and delete keys; suspend and resume agents |
gatewayAi.budgets:read | View Budgets & Limits, the Hierarchy and a key's effective limits |
gatewayAi.budgets:admin | Create, edit and delete budgets and org units, move keys between org units, issue a key into an org unit |
gatewayAi.models:read | View provider credentials (masked), model groups and deployments |
gatewayAi.models:admin | Add, change, rotate, move and delete credentials; manage model groups, deployments, routing, fallbacks and cache |
gatewayAi.settings:read / gatewayAi.settings:admin | View or change Project Settings → Budget AI |
tenantSecurity.gatewayKeys:admin | Block and unblock keys, and suspend and resume agents, during an incident, without any other Budget AI access |
Built-in roles:
| Role | Budget AI access |
|---|---|
| Owner | Admin on every page |
| Department Admin | Admin on every page |
| Security & Compliance | Read on every page, plus block and unblock through tenant security |
| Cost Analyst | Read on every page |
| Auditor | Read on every page |
| Manager Viewer, End User, None | No access |
To give someone a narrower set, for example keys only, create a custom role and set the Budget AI pages individually.
In this section
For capabilities that are planned but not yet available, see the Roadmap.