Docs/Aigovernance/Violations

AI Violations

AI Violations lists every place where AI Guardrails found something: which agent, which category, which values, and in which trace. Rows come from both guardrail modes:

  • Scheduled Detection results from the PII, PCI, PHI, Ethics & Bias and Safety scans, shown with source Platform.
  • Scheduled Detection results from custom detectors, Secrets & Credentials and Source Code, shown with source Scheduled.
  • Real-time Enforcement checks that found a violation, shown with source SDK (Python SDK and REST API) or Connector (Portkey connector and Budget AI gateway).
ℹ

Enterprise feature. Open it from Governance → AI Violations. It appears when AI Violations is turned on in Organization Settings → Features. The list contains flagged prompt and response content, so it requires the PII data permission (pii:read) in Roles & Access; among the built-in roles only Owner and Security & Compliance have it. The detected values themselves are hidden from users who are not allowed to see trace input and output.

Members who own agents see the violations of their own agents only, including the flagged content, without the PII data permission ("You see violations of the agents you own only."). They cannot mark detections or open the Detector accuracy report. See Agent owners.

A banner at the top says what needs attention across all time: agents with violations and no guardrail (Add guardrails), agents with violations that are not blocked (Turn on blocking), and one agent causing half or more of all detections. Select a line to open the agents in a side panel with a next step, which appears only when you can open the AI Guardrails policy it leads to.

The page has two tabs: Detections (the default, described below) and Drift and suspend activity.

What each row shows

ColumnDescription
TimestampWhen the trace or check happened
NameThe agent. For Platform rows it links to the trace.
PII, PCI, PHI, Ethics/Bias, SafetyWhich categories were found
DetectionsThe labels that were found, for example email, credit card number, a harm category, secrets.aws_access_key, sourceCode.python or Custom: <label> for a custom detector
Detected AtWhen the detection ran
SourcePlatform (Scheduled Detection of PII, PCI, PHI, Ethics & Bias and Safety), Scheduled (Scheduled Detection of custom detectors, Secrets & Credentials and Source Code), SDK (Real-time Enforcement through the Python SDK or the REST API) or Connector (Real-time Enforcement through the Portkey connector or the Budget AI gateway)
ResultFor SDK and Connector rows: Pass (Audit Only), Warning, Blocked or Sanitized. For Scheduled rows: Warning or Pass, because a scheduled scan never blocks or changes anything.
WebhookWhether a policy violation webhook went out for the detection: Sent, Pending or Failed, with the number of deliveries when there were several and the breakdown in a tooltip, or "-" when none was sent

Filtering

  • Filter: All Detections, PII, PCI, PHI, Ethics/Bias, Safety, Custom (custom detector matches), Secrets & Credentials, Source Code or Tools (tool calls that broke an agent's tool permissions: not approved, expired, not allowed for the user, outside an argument rule, carrying sensitive data or over the rate limit). Tool violations from the ANTS SDKs have source SDK, and those from the Budget AI gateway and the Portkey connector have source Connector. The result is Blocked when the agent's policy blocked the call and Pass when it was only recorded.
  • Agent: all agents or one agent.
  • Time range: Last 24 hours, Last 7 days (default), Last 14 days, Last 30 days, or All time.

You can also open the list already filtered from other views, for example from an agent's governance profile.

Custom detector rows

The Custom filter lists the checks and scheduled scans in which a custom detector matched. The Detections column shows each detector that matched as Custom: <label>, where the label is the detector's Label on detections, and Detection Details lists them under Custom detectors. Real-time matches are shown with source SDK or Connector, and scheduled matches with source Scheduled. A scheduled match has the result Warning for detectors set to Block, Redact or Warn, and Pass for Audit Only detectors, because a scheduled scan never blocks or changes anything.

The matched values are not stored, so the list shows which detector matched, not what it matched. The same label is in the guardrail API result, and in policy violation webhooks and SIEM push alerts.

Secrets and source code rows

The Secrets & Credentials filter lists the checks and scheduled scans in which a Secrets & Credentials rule matched, and the Source Code filter those in which Source Code found code. These categories have no column of their own: the Detections column shows the rule labels, such as secrets.aws_access_key or sourceCode.python. The matched secrets and code are not stored with the detection, so there is no value to highlight.

Scheduled rows for these categories have source Scheduled and the result Pass. Real-time rows have the result the policy's action produced, for example Sanitized for a secret masked by Redact or Blocked for source code under Block.

Rule-based card number and SSN findings (see Card number and SSN checks) are listed under PCI and PII with the labels pci.card_number, pci.test_pan and pii.us_ssn.

Violation details

Select a row to open Detection Details:

  • Each detected PII, PCI or PHI value with its label and confidence score, highlighted where it appears in the input or the output. The values are masked: only their shape and last four letters or digits show, for example •••• •••• •••• 1234, and values of four characters or fewer are fully masked. Select Show detected values to see them for that row. Each reveal is written to the audit log (resource type trace, action revealDetectionValues) and needs pii:read. The button is not shown when trace input and output are hidden for your organization or project.
  • Add to case, for rows that come from a trace, adds the trace to an investigation case.
  • For Ethics & Bias and Safety, the category that was flagged, for example S10 Hate Speech, Gender Bias, or Jailbreak / Prompt Injection.
  • For custom detector matches, the detectors that matched, under Custom detectors.
  • For SDK, Connector and Scheduled rows, the result, the direction (input or output) and the risk level.
  • For Platform rows, View full trace details opens the complete trace.
  • When Detection review is on, Review detections, with buttons to mark each label.

Review detections

Analysts record whether a detection was right, so noisy detectors can be tuned from evidence. Review is off by default.

Turn on Detection review

An Owner or a Security & Compliance member turns on Detection review in Organization Settings → Features. While it is off, the review buttons and the Detector accuracy report are hidden, and marks cannot be saved. Organizations that leave it off see no change.

Mark a detection

Open the detection

In AI Violations, select a row that has PII, PCI or PHI detections. Detection Details opens.

Find Review detections

Under Review detections, each detected label is listed once, for example email or credit card number.

Mark the label

Select False positive or Real detection next to the label. The selected button stays highlighted. To clear a mark, select the highlighted button again.

A mark applies to the label on that row, not to each value: if a row found three email addresses, one mark covers email for that row. Selecting the other button changes the mark. Each mark records who set it, and every mark, change and clear is written to the audit log.

Review covers PII, PCI and PHI labels. Ethics & Bias, Safety, Secrets & Credentials, Source Code and custom detector results cannot be marked yet, and a mark does not suppress future detections.

Who can mark

Marking needs the PII data permission (pii:read), because the analyst must be able to see the flagged content to judge it. Among the built-in roles, Owner and Security & Compliance have it.

Detector accuracy report

When Detection review is on, the Detector accuracy card at the top of AI Violations shows, for each PII, PCI and PHI label:

ColumnDescription
DetectorThe detection label, for example email
Flagged tracesTraces in which Scheduled Detection found the label in the selected window
ReviewedMarks set or changed in the window, False positive and Real detection together
False positivesMarks set to False positive in the window
False-positive rateFalse positives divided by Reviewed. It shows "-" until someone has reviewed the label.

Select 7d, 30d (default) or 90d to change the window. The report covers the whole project; the list's Filter, Agent and Time range do not change it.

The rate measures accuracy among the detections analysts reviewed, not across all traffic, so review a representative sample before acting on it. Flagged traces counts Scheduled Detection results only; marks you set on Real-time Enforcement rows are counted in Reviewed and False positives. Use the report to find labels that need a higher confidence threshold or a narrower label selection in AI Guardrails.

The report needs the AI Violations permission at Read. Because it is on the AI Violations page, viewers also need pii:read, so in practice it is available to Owner and Security & Compliance.

Scheduled detection

Each agent's policy in AI Guardrails sets its own schedule: Hourly, Daily or Weekly. Each run scans the agent's traces received since the previous run, for the categories and labels selected in that policy and its custom detectors. Change the schedule, labels or confidence threshold in AI Guardrails.

Incidents

Scheduled detections open incidents automatically when these integrations are set up:

  • ServiceNow: when the ServiceNow integration is connected with incident creation turned on, PII, PCI, PHI, ethics and bias, and jailbreak detections open ServiceNow incidents.
  • PagerDuty: when the PagerDuty integration is connected with incident creation turned on, and PagerDuty Alerting is on in the agent's policy, detections open incidents on the selected PagerDuty service with the selected severity.

Real-time Enforcement records appear in this list but do not open incidents. To send violations to a security tool instead, use policy violation webhooks or SIEM push.

⚠

Violations can contain the actual sensitive values that were detected. Limit the pii:read permission to the people who need it.

Drift and suspend activity

Select the Drift and suspend activity tab to see, in one list, the project's drift alerts and every time an agent was suspended or resumed.

  • Choose All, Drift or Suspend and resume, and a window of 7d, 30d (default) or 90d.
  • Each row shows When, Type (Drift, Suspended or Resumed), Agent, Details, Severity, Notified and By. Details is the alert's description, or for a suspension or resume the number of keys blocked or unblocked. Notified shows where a drift alert was sent and when, or Not sent. By is the person who suspended or resumed the agent.
  • Up to 100 rows are shown, newest first.
  • Automatic suspensions by behaviour checks are not listed here; find them in the audit log under action autoSuspend.

The view needs the AI Violations permission at Read.

Next steps

© 2026 ANTS Platform, Inc.Docs v1.0 · Last updated June 2026