Guardrails
Budget AI can check traffic inline with the same policies as AI Guardrails. When guardrail checks are on, the gateway checks each request's prompt before it reaches the provider, and each non-streamed response before it reaches the caller. Agents and applications get this with no code change: they keep calling the gateway with their Budget AI key.
Guardrail checks are off by default. You turn them on for a project or for individual keys.
What is checked
| Traffic | Checked | When |
|---|---|---|
| The input of each request | Yes | After the key is accepted and before the provider is called |
| Non-streamed responses | Yes | After the provider answers and before the response is returned |
| Streamed responses | No | The prompt of a streamed call is checked; the streamed response is not checked inline |
Checks use the detection categories and labels of the policy: PII, PCI and PHI entities, Ethics & Bias, and Safety (jailbreak and prompt injection). See Detection categories. Stored traffic can still be scanned on a schedule with Scheduled Detection.
Which policy applies
The gateway applies the Real-time Enforcement policy of the agent the key is bound to. A key that is not bound to an agent uses the Default guardrail policy chosen in Project Settings → Budget AI, which is the Real-time Enforcement policy of an agent you pick.
Policies are created and edited in Governance → AI Guardrails, and a change applies to gateway traffic and SDK traffic alike. If no policy applies to a key, its requests are served without checks.
Actions
The policy's action decides what the gateway does when a check finds a violation:
| Policy action | On the prompt | On a non-streamed response |
|---|---|---|
| Audit Only | The violation is recorded and the request is sent unchanged | The violation is recorded and the response is returned unchanged |
| Block | The request is refused with 400 guardrail_blocked and never reaches the provider | The response is withheld and the caller receives 400 guardrail_blocked |
| Redact | Detected PII, PCI and PHI values are masked and the masked prompt is sent. Ethics & Bias and Safety violations are blocked. | Detected values are masked in the returned response. Ethics & Bias and Safety violations are blocked. |
A blocked call returns the policy's Blocked Response Message in the OpenAI error envelope, so OpenAI SDKs raise their usual exception. See Errors.
Every check that finds a violation is recorded in AI Violations, and the gateway trace records the guardrail result.
Failure behaviour
Choose what happens when the detection service cannot be reached:
| Setting | Behaviour |
|---|---|
| Fail open | The request is served without the check |
| Fail closed | The request is refused with 503 guardrail_unavailable and nothing reaches the provider |
Set it for each policy the gateway uses, in the guardrail settings of Project Settings → Budget AI.
Turn on guardrail checks
Check the detection services
Guardrail checks use the same detection services as AI Guardrails, which are enabled per deployment. If Governance → AI Guardrails is not available, ask your ANTS administrator.
Create or review the policy
In Governance → AI Guardrails, make sure the agent has a policy in Real-time Enforcement mode with the categories and action you want.
Turn on checks for the project
In Project Settings → Budget AI, turn on Guardrail checks, choose the Default guardrail policy for keys not bound to an agent, and choose Fail open or Fail closed for each policy the gateway uses. This needs gatewayAi.settings:admin.
Override per key where needed
On a key's Advanced settings, Guardrail checks is Project default, On or Off. Use it to turn checks on for some keys only, or to exempt a key. This needs gatewayAi.keys:write.
Changes to the switch, the default policy and the failure setting are written to the audit log.
Checks add the time the detection service takes to each checked call. Turn them on for the keys that carry the traffic you need to inspect.
Budget AI in front of another gateway
If your agents already call models through another gateway, such as Portkey, you can place Budget AI in front of it for agent traffic. Add the other gateway in Providers & Credentials as an OpenAI-compatible provider. Agents then hold Budget AI keys, so guardrail checks, key expiry and agent suspension take effect before traffic reaches the other gateway.
To keep Portkey as your only gateway instead, use the Portkey connector, which runs the same checks as a Portkey guardrail.
Related
- AI Guardrails: policies, detection categories and the SDK.
- Keys: binding keys to agents and per-key settings.
- Security: fail-closed behaviour and access control.