Providers & Credentials
A provider credential is the account Budget AI uses to call a provider on your behalf: a provider family, an optional base URL, and a secret. Credentials belong to a project and are managed in Budget AI → Models & Routing → Provider credentials. Applications never see them; they hold only Budget AI keys.
Supported provider families
Every family accepts the same OpenAI-style request from your application. Budget AI translates it for the provider and translates the answer back, including streamed responses.
| Family | Base URL | Also needed | Secret | Embeddings |
|---|---|---|---|---|
| OpenAI-compatible | Optional. Default https://api.openai.com/v1. Set it for any other OpenAI-compatible API. | None | API key, sent as Authorization: Bearer | Yes |
| Azure OpenAI | Required: your resource endpoint | API version (optional, default 2025-02-01-preview) | API key, sent as api-key | No |
| Anthropic | Optional. Default https://api.anthropic.com | None | API key, sent as x-api-key | No |
| AWS Bedrock | Optional. Default https://bedrock-runtime.<region>.amazonaws.com | Region (required) | Access key JSON: {"accessKeyId": "...", "secretAccessKey": "..."} | No |
| Google Vertex AI | Optional. Default is the regional Vertex AI endpoint, or the Gemini API when you use a Gemini API key | Region (required) | Service account JSON, or a Gemini API key | No |
Notes per family:
- OpenAI-compatible requests go to
{base URL}/chat/completionsand{base URL}/embeddings. Streamed requests ask the provider for a final usage chunk, so streamed calls are costed from real usage. For Portkey, the secret is sent asx-portkey-api-keyinstead; see Portkey as an upstream. - Azure OpenAI requests go to
{base URL}/openai/deployments/{provider model}/chat/completions?api-version=.... Use the Azure deployment name as the provider model. A base URL that already ends in/openai/deploymentsis not repeated. - Anthropic requests are translated to the Messages API. When the caller sets no
max_tokens, Budget AI sends 4096. - AWS Bedrock requests use the Bedrock Converse API (and its streaming variant), signed with AWS Signature Version 4.
- Google Vertex AI requests use
generateContentandstreamGenerateContent. With a service account, Budget AI obtains and refreshes the access token itself.
A model group can mix families. For embeddings, models whose family has no embeddings API are skipped, so a mixed group still serves embeddings from its OpenAI-compatible models. See Model Groups & Routing.
Add a credential
Open Provider credentials
Go to Budget AI → Models & Routing and select Add credential under Provider credentials. You need gatewayAi.models:admin.
Fill in the form
Enter a Name (unique in the project), choose the Provider family, and fill in Base URL, API version or Region where shown. Paste the Secret.
Confirm with step-up verification
Creating or changing a credential asks for step-up verification.
The secret is never shown again. The credentials table shows the last four characters, when it was last rotated, where it is stored, and how many models use it.
Import an existing LLM Connection
If the project already has an LLM Connection for OpenAI, Azure OpenAI or Anthropic, select Import LLM Connection, choose the connection and give the credential a name. Budget AI copies the connection's encrypted secret, base URL and family without decrypting it. Imported credentials are marked imported.
Portkey as an upstream
If your agents already call models through Portkey, Budget AI can sit in front of it: agents hold Budget AI keys, and Budget AI forwards their traffic to Portkey, which calls the providers. Budget AI's model access, budgets, key expiry, key blocking and inline guardrails apply before traffic reaches Portkey, and Portkey's own logs and guardrails still apply after.
Portkey authenticates with an x-portkey-api-key header rather than a bearer token. An OpenAI-compatible credential sends its secret that way, with no Authorization header, when its base URL is on portkey.ai, or when Portkey gateway is turned on in the credential form.
Add the credential
In Budget AI → Models & Routing, select Add credential and choose OpenAI-compatible.
- Base URL: your Portkey address, for example
https://api.portkey.ai/v1. - Portkey gateway: not needed for
portkey.aiaddresses, which are recognised automatically. Turn it on for a self-hosted Portkey, or to set a Portkey provider. - Portkey provider (optional): shown when Portkey gateway is on. It is sent as
x-portkey-provider, for example@openai-prod. - Secret: your Portkey API key.
Complete step-up verification.
Add models
Add deployments to a model group with this credential. If you set a Portkey provider, use the provider's model name, for example gpt-4o. Otherwise name the provider in each model, for example @openai-prod/gpt-4o, which Portkey resolves.
Check a request
Send a request through Budget AI with an x-portkey-trace-id header and find it in Portkey's logs under that trace id. A wrong Portkey key returns Portkey's own 401 through Budget AI.
Health checks use the same headers.
Headers passed through to Portkey
Callers can send these headers to Budget AI on chat completions. They are passed on only to Portkey upstreams:
| Header | Passed when |
|---|---|
x-portkey-metadata | It is a flat JSON object whose values are strings, numbers or booleans, up to 4,096 characters |
x-portkey-trace-id | It is up to 128 characters of letters, digits, ., _, : and - |
A header that does not meet these rules is dropped and the request goes ahead without it.
These are never passed from the caller:
x-portkey-api-key: Portkey always receives the credential's own key.x-portkey-config: a Portkey config can switch the provider, model or virtual key, which would route around the model groups the Budget AI key is allowed to call and the budgets attached to them.
If the Portkey connector also runs AI Guardrails checks on the same traffic, content is inspected twice. Turn off one of the two: Budget AI inspection or the connector's guardrail.
Base URL rules
A base URL receives the credential's secret, so Budget AI refuses one that:
- is not a valid
http://orhttps://URL; - embeds a username or password;
- points at a loopback, private, carrier-grade NAT or link-local address,
localhost, or a cloud metadata host.
Trailing slashes are ignored. Leave the field empty to use the family's default.
Where provider secrets are stored
Each credential's secret is stored in one of two places. The Stored in column of the credentials table shows which.
| Store | How it works |
|---|---|
| Database | The secret is encrypted with AES-256-GCM under the platform's ENCRYPTION_KEY and stored in the platform database. This is the default. |
| AWS Secrets Manager | The secret is written to AWS Secrets Manager at <prefix>/<project id>/<credential id> and the database holds only that reference. The path is derived by the platform and is never accepted as input. |
The store for new credentials is an instance setting (GATEWAY_SECRETS_STORE=aws, see Self-Hosting & Configuration). Project Settings → Budget AI → Provider secret storage shows which store is in use, whether the managed store is reachable, and how many credentials are still in the database.
When the managed store is configured, credentials still in the database show a Move action. Moving writes the secret to AWS Secrets Manager, reads it back to confirm, and only then switches the credential to the new reference. Step-up verification is required.
Gateway replicas keep provider secrets in memory and refresh secrets held in AWS Secrets Manager every five minutes by default. If a secret cannot be read, a replica keeps the copy it already had. A credential with no readable secret is left out of routing until it can be read.
Rotate a provider secret
Budget AI cannot create or revoke keys at the provider. Rotation is a safe swap with a check in the middle:
Create a replacement key in the provider's console
Paste it into Budget AI
Select Rotate on the credential and paste the New provider secret. Budget AI stores it, reads it back, and switches every gateway replica to it. Traffic keeps flowing while it propagates. Step-up verification is required.
Wait for traffic to be served with it
The rotate panel shows Since the last rotation: how many of the credential's models have served or failed a request since the swap.
Revoke the old key in the provider's console
The credentials table shows when each secret was last rotated, or never rotated.
Provider spend caps
A leaked provider secret is bounded only by a spend cap set at the provider. The credentials table marks credentials with no cap recorded as a reminder to set one in the provider's console.
Change or delete a credential
- Edit changes the name, base URL, API version or region, and optionally the secret (leave New secret empty to keep it). Step-up verification is required.
- Delete is refused while any model uses the credential. Remove or re-point those models first. When a credential held in AWS Secrets Manager is deleted, its managed secret is scheduled for deletion with a recovery window.
Every create, change, rotation, move, import and delete is written to the audit log.
Health checks
Health checks are off by default. Turn them on per project in Project Settings → Budget AI → Health checks, then choose a mode and How often (every minute, 5 minutes, 15 minutes or hour).
| Mode | What it does | Cost |
|---|---|---|
| Check the credential only | Asks the provider to list models with the credential. Catches a dead or revoked key, not a withdrawn model. | Free |
| Send a one-token request | Sends a one-token completion to each exact model. Catches a dead key and a dead model. | Billable for every model on every check, whether or not anyone is using it |
Health checks run inside the gateway, cover every enabled model in the project, use no Budget AI key, create no trace and never count against limits or budgets. The result shows as each model's status in Models & Routing and on the Model health card of the Overview. A model's status is the newer of its last health check and its last real request.
Health checks report status only. Taking a failing model out of rotation is done by benching, which is driven by real traffic. See Model Groups & Routing.