Docs/Core Concepts/Three Pillars

Three Pillars of LLMOps

AgenticAnts implements LLMOps (Large Language Model Operations) through three foundational pillars that work together to provide comprehensive AI operations management.

Overview

LLMOps is the overarching discipline that encompasses the entire lifecycle of LLM operations from development to production. Our three pillars approach ensures complete coverage of AI operational needs:

LLMOps Framework

LLMOps provides the comprehensive framework for managing LLM operations:

  • Model Lifecycle Management - Selection, versioning, deployment, and retirement
  • Prompt Operations - Prompt engineering, versioning, and optimization
  • Performance Optimization - Latency, throughput, and cost optimization
  • Model Governance - Policies, compliance, and risk management
  • Versioning & Deployment - CI/CD pipelines and rollback strategies

Learn more about LLMOps →

Pillar 1: AI Cost (FinOps)

Cost Visibility, Allocation, Optimization & Accountability

What is AI Cost (FinOps)?

Cost (FinOps) for AI helps organizations understand, control, and optimize AI spending. Cost is the primary indicator and measurable outcome of FinOps - providing:

  • Cost Visibility: See where every dollar is spent in real-time
  • Cost Attribution: Track costs by customer, team, or product
  • Cost Optimization: Identify and eliminate waste
  • Cost Accountability: Budget allocation and forecasting

Key Capabilities

Token Usage Monitoring

Track every token consumed by your AI systems:

typescript
// Wrap the OpenAI client once; every call is auto-traced const openai = observeOpenAI(new OpenAI()); const response = await openai.chat.completions.create({ model: "gpt-4", messages: [{ role: "user", content: query }], }); // AgenticAnts records on each traced generation: // - Model used: gpt-4 // - Tokens: prompt=150, completion=200, total=350 // - Cost: $0.0105 (based on current pricing)

Cost Per Customer Query

Understand the economics of your AI operations. Cost breakdowns by customer, team, or product are available as dashboard reports at https://app.agenticants.ai. Attribute costs by attaching metadata, tags, or a user/session ID to your traces, then slice and filter spend in the dashboard. There is no programmatic metrics/cost query API in the SDK.

Budget Management

Set budgets and configure spend alerts directly in the dashboard at https://app.agenticants.ai. Define quarterly or monthly budgets and warning/critical thresholds there; alerting is a dashboard feature and is not configured through the SDK.

ROI Analytics

Measure the business impact of AI investments using the analytics dashboards at https://app.agenticants.ai. ROI, cost-per-conversion, and customer-lifetime-value views are built from your ingested trace and cost data; there is no programmatic analytics query API in the SDK.

AI Cost Best Practices

  1. Tag Everything: Use consistent tagging for cost attribution
  2. Set Budgets: Define spending limits for teams and projects
  3. Monitor Regularly: Review costs weekly, not monthly
  4. Optimize Models: Use smaller models where appropriate
  5. Cache Responses: Reduce redundant LLM calls

Learn more about AI Cost (FinOps) →

Pillar 2: AI Resilient (SRE)

Latency, Performance, Availability & Reliability

What is AI Resilient (SRE)?

Resilient (SRE) for AI systems emphasizes latency, performance, and operational health through SRE principles:

  • Low Latency: Fast response times and optimized performance
  • High Performance: Maximum throughput and efficiency
  • Reliable: High availability and fault tolerance with error budgets
  • Observable: Complete visibility into system behavior
  • SLAs/SLOs: Service level objectives and compliance

Key Capabilities

End-to-End Tracing

Follow requests through your entire AI stack. With tracing configured once at startup, wrap functions with observe or open spans with startObservation to build the execution tree:

typescript
const handleQuery = observe( async (query: string) => { const span = startObservation("span", { name: "retrieve-customer-context" }); const context = await getCustomerContext(query); span.end(); const gen = startObservation("generation", { name: "llm-inference" }); const answer = await callModel(query, context); gen.end(); return answer; }, { name: "customer-support-query" }, );

The dashboard then visualizes the complete execution path:

text
# Trace shows complete execution path (as visualized in the dashboard) Trace: customer-support-query (2.3s) ├─ Span: input-validation (10ms) ├─ Span: retrieve-customer-context (150ms) │ └─ Span: database-query (145ms) ├─ Span: vector-search (200ms) │ ├─ Span: embedding-generation (50ms) │ └─ Span: similarity-search (150ms) ├─ Span: llm-inference (1.8s) │ ├─ Span: prompt-construction (5ms) │ ├─ Span: api-call (1.78s) │ └─ Span: response-parsing (15ms) └─ Span: response-formatting (140ms)

Performance Monitoring

Track key performance metrics in the dashboards at https://app.agenticants.ai. Latency percentiles (p50/p95/p99), error rate, and throughput are computed from your ingested traces and shown there; the SDK does not expose a metrics/trends query API. For example, a 24-hour view might surface p50 latency of 1,200ms, p95 of 3,500ms, p99 of 5,200ms, a 0.5% error rate, and 45 req/s throughput.

Automated Alerting

Get notified when things go wrong by configuring alerts in the dashboard at https://app.agenticants.ai. Create conditions such as "error rate > 5% over 5m" or "p95 latency > 5000ms over 10m" and route them to channels like Slack, PagerDuty, or email. Alerting is a dashboard feature; there is no programmatic alerts API in the SDK.

Incident Response

Quickly diagnose and resolve issues from the dashboard at https://app.agenticants.ai. Incident timelines, root-cause analysis, and related-incident views are dashboard features built on your ingested traces and are not exposed through the SDK.

AI Resilient Best Practices

  1. Set SLOs: Define Service Level Objectives for latency and availability
  2. Monitor Proactively: Don't wait for users to report performance issues
  3. Automate Responses: Auto-remediate common issues
  4. Learn from Incidents: Conduct post-mortems
  5. Test Resilience: Implement chaos engineering

Learn more about AI Resilient (SRE) →

Pillar 3: AI Governance

Compliance, Risk Management & Policy Enforcement

What is AI Governance?

AI Governance provides comprehensive oversight of AI/LLM operations through governance, compliance, risk management, and security:

  • AI Governance: Policy enforcement and model usage controls
  • Compliance: Meet regulatory requirements (SOC2, GDPR, HIPAA, AI Act)
  • Risk Management: Continuous assessment and mitigation
  • Data Protection: Prevent sensitive data leaks
  • Access Control: Manage who can access what with RBAC
  • Audit Trails: Complete logs for compliance and forensics

Key Capabilities

PII Detection & Protection

Automatically identify and protect sensitive data:

typescript
// Capture the request as a span; AgenticAnts scans ingested traces for PII const span = startObservation("span", { name: "customer-query" }); span.update({ input: "My SSN is 123-45-6789 and email is john@example.com" }); span.end(); // In the dashboard you then see: // - PII detected: SSN, Email // - Sensitive values redacted in storage // - Security alert raised // - Audit log entry created

Security Guardrails

Prevent harmful or policy-violating outputs:

python
from ants_platform import AntsGuardrailsClient guardrails = AntsGuardrailsClient( public_key="pk_...", secret_key="sk_...", host="https://api.agenticants.ai", ) # Screen model output against your policies before returning it result = guardrails.check(input=query, output=agent_output) if result.blocked: raise ValueError("Response blocked by guardrail policy")

Policy rules (such as PII redaction, toxic-content blocking, or financial-advice warnings) are configured for your project in the dashboard at https://app.agenticants.ai and enforced by the guardrails client.

Compliance Reporting

Generate compliance reports from the dashboard at https://app.agenticants.ai. SOC2/GDPR/HIPAA reporting, control evidence, and per-user data exports are dashboard features and are not exposed through the SDK.

RBAC & Access Control

Fine-grained permissions management is handled in the dashboard at https://app.agenticants.ai. Define roles, scope permissions to projects, and assign roles to users there; role and user administration is not exposed through the SDK.

Audit Trails

Complete logging of all activities is available in the dashboard at https://app.agenticants.ai. Browse and filter audit events (by action, user, resource, IP, status, and date range) there; audit logs are a dashboard feature and are not queried through the SDK.

AI Governance Best Practices

  1. Principle of Least Privilege: Give minimum necessary access
  2. Policy Enforcement: Define and enforce AI/LLM usage policies
  3. Regular Audits: Review access, compliance, and activities regularly
  4. Risk Assessment: Continuously assess and mitigate AI risks
  5. Incident Response Plan: Have a plan before incidents occur

Learn more about AI Governance →

Integration of the Three Pillars

The pillars work together to provide comprehensive coverage:

Example: Production Incident

Example: Cost Optimization

Getting Started with Each Pillar

Start with AI Cost

Track costs and optimize spending

Start with AI Resilient

Monitor latency and improve performance

Start with AI Governance

Govern AI, manage compliance and risk

© 2026 ANTS Platform, Inc.Docs v1.0 · Last updated June 2026