Docs/Core Concepts/Tracing

Tracing Fundamentals

Distributed tracing is essential for understanding AI agent behavior. Learn how AgenticAnts implements tracing for AI systems.

What is Tracing?

Tracing tracks a request as it flows through your system, capturing:

  • What happened
  • When it happened
  • How long it took
  • What data was involved
  • Any errors that occurred

Trace vs Span vs Event

Example:

Creating Traces

Basic Trace

First, configure tracing once at startup (e.g. in an instrumentation.ts that is imported before anything else). Tracing requires Node.js 20+.

typescript
// instrumentation.ts import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node' import { AntsPlatformSpanProcessor } from '@antsplatform/otel' import { setAntsPlatformTracerProvider } from 'ants-platform' const provider = new NodeTracerProvider({ spanProcessors: [ new AntsPlatformSpanProcessor({ publicKey: process.env.ANTS_PLATFORM_PUBLIC_KEY, secretKey: process.env.ANTS_PLATFORM_SECRET_KEY, baseUrl: 'https://api.agenticants.ai', }), ], }) provider.register() setAntsPlatformTracerProvider(provider)

Then instrument your agent code. startActiveObservation opens a trace-level span; updateActiveTrace and updateActiveObservation attach input, output, and metadata.

typescript
import { startActiveObservation, updateActiveTrace, updateActiveObservation, } from 'ants-platform' async function processQuery(query: string) { return startActiveObservation('span', { name: 'process-customer-query' }, async () => { updateActiveTrace({ name: 'process-customer-query', input: query, userId: 'user_123', metadata: { timestamp: new Date().toISOString() }, }) try { // Your agent logic const result = await agent.process(query) updateActiveObservation({ output: result, metadata: { success: true, confidence: 0.95 }, }) return result } catch (error) { updateActiveObservation({ level: 'ERROR', statusMessage: error.message, metadata: { stack: error.stack }, }) throw error } }) }

Nested Spans

Add spans for detailed breakdown:

Each startObservation returns a span you must .end(). Use type "generation" for LLM calls so token/cost data is captured.

typescript
import { startObservation } from 'ants-platform' async function processQuery(query: string) { const trace = startObservation('span', { name: 'process-query' }) // Span 1: Classification const classifySpan = startObservation('span', { name: 'classify-intent' }) const intent = await classifyIntent(query) classifySpan.update({ output: { intent } }).end() // Span 2: Retrieval const retrievalSpan = startObservation('span', { name: 'retrieve-context' }) const context = await retrieveContext(intent) retrievalSpan.update({ output: { documents: context.length } }).end() // Span 3: Generation const generationSpan = startObservation('generation', { name: 'generate-response' }) const response = await generateResponse(query, context) generationSpan .update({ output: response.text, usage: { total: response.usage.total }, metadata: { cost: response.cost }, }) .end() trace.update({ output: response.text }).end() }

Trace Context

Propagation

The SDK is built on OpenTelemetry, so trace context propagates automatically through any code running inside an active observation. Nested observations are linked to their parent without manual ID plumbing:

typescript
await startActiveObservation('span', { name: 'main-request' }, async () => { // Any observation started here is automatically a child of 'main-request'. const span = startObservation('span', { name: 'downstream-operation' }) // ... do work ... span.end() })

For propagation across separate services or processes, use the standard W3C traceparent header from @opentelemetry/api so the receiving service continues the same OpenTelemetry trace.

Correlation

Group related traces under a single session by setting a session ID on each trace. Filter and replay full sessions from the dashboard at https://app.agenticants.ai:

typescript
async function runQuery(name: string, sessionId: string) { return startActiveObservation('span', { name }, async () => { updateActiveTrace({ sessionId }) // ... query logic ... }) } await runQuery('query-1', 'session_xyz') await runQuery('query-2', 'session_xyz')

Metadata and Tags

Adding Metadata

Enrich traces with context:

typescript
await startActiveObservation('span', { name: 'agent-execution' }, async () => { updateActiveTrace({ // First-class trace fields userId: 'user_123', sessionId: 'session_xyz', metadata: { // User information userEmail: 'user@example.com', userTier: 'premium', // Request context requestId: 'req_abc', ipAddress: '192.168.1.1', // Business context customerId: 'customer_456', accountType: 'enterprise', feature: 'customer-support', // Technical context agentVersion: '1.2.3', model: 'gpt-4', temperature: 0.7, region: 'us-east-1', }, }) // ... agent logic ... })

Using Tags

Categorize and filter traces:

python
from ants_platform import observe, get_client @observe(name='agent-execution') def run_agent(): client = get_client() client.update_current_trace( tags=['production', 'customer-success', 'high', 'variant_b'], ) # ... agent logic ...

Filtering and querying traces by tag is a dashboard feature. Browse and filter traces by tag at https://app.agenticants.ai.

Sampling Strategies

Head-Based Sampling

Because the SDK is built on OpenTelemetry, head-based sampling is configured with a standard OTel sampler on the NodeTracerProvider. Use TraceIdRatioBasedSampler to keep a fixed fraction of traces:

typescript
const provider = new NodeTracerProvider({ sampler: new TraceIdRatioBasedSampler(0.1), // Sample 10% of traces spanProcessors: [ new AntsPlatformSpanProcessor({ publicKey: process.env.ANTS_PLATFORM_PUBLIC_KEY, secretKey: process.env.ANTS_PLATFORM_SECRET_KEY, baseUrl: 'https://api.agenticants.ai', }), ], }) provider.register() setAntsPlatformTracerProvider(provider)

For custom per-request logic, gate whether you open an observation at all:

typescript
const shouldSample = (request) => { if (request.expectedError) return true // Always sample errors if (request.userTier === 'premium') return true // Always sample premium users return Math.random() < 0.1 // Sample 10% of others } async function handle(request) { if (!shouldSample(request)) return process(request) return startActiveObservation('span', { name: 'agent-run' }, () => process(request)) }

Tail-Based Sampling

Tail-based sampling decides whether to keep a trace after it completes, based on its outcome (errors, latency, customer tier). In the Python SDK you implement this by deciding whether to flush a finished trace. A common pattern is to mark traces you want to keep with a tag, then apply your own keep/drop logic before client.flush():

python
from ants_platform import observe, get_client def keep_trace(error: bool, duration_ms: int, customer_tier: str) -> bool: if error: # Keep all errors return True if duration_ms > 5000: # Keep all slow requests return True if customer_tier == 'enterprise': # Keep 50% of high-value customers return random.random() < 0.5 return random.random() < 0.1 # Keep 10% of everything else @observe(name='agent-run') def run_agent(request): client = get_client() # ... agent logic, measuring error/duration ... if keep_trace(error, duration_ms, request.customer_tier): client.update_current_trace(tags=['sampled'])

Performance Tracking

Measuring Latency

typescript
// Automatic timing: duration is the wall-clock time between start and end const span = startObservation('generation', { name: 'llm-call' }) const result = await llm.generate(prompt) span.end() // Duration calculated automatically // You can also record your own measurement as metadata const start = Date.now() const opResult = await operation() const opSpan = startObservation('span', { name: 'operation' }) opSpan.update({ metadata: { durationMs: Date.now() - start } }).end()

Token Tracking

The simplest way to capture tokens and cost is the OpenAI drop-in, which records usage automatically. Just swap the import:

python
# Replace `import openai` with the instrumented drop-in from ants_platform.openai import openai response = openai.chat.completions.create( model='gpt-4', messages=messages, ) # Prompt/completion tokens and cost are captured automatically.

If you manage the generation span yourself, pass usage_details:

python
from ants_platform import get_client client = get_client() with client.start_generation(name='llm-inference', model='gpt-4') as span: response = openai.chat.completions.create(model='gpt-4', messages=messages) span.update( output=response.choices[0].message.content, usage_details={ 'input': response.usage.prompt_tokens, 'output': response.usage.completion_tokens, 'total': response.usage.total_tokens, }, )

Error Tracking

Recording Errors

typescript
const span = startObservation('span', { name: 'risky-operation' }) try { const result = await riskyOperation() span.update({ output: result }).end() } catch (error) { span .update({ level: 'ERROR', statusMessage: error.message, metadata: { stack: error.stack, code: error.code, operation: 'riskyOperation', }, }) .end() throw error }

Error Categories

python
# Classify errors and record them on the current span. # The SDK level is one of DEBUG / DEFAULT / WARNING / ERROR. if isinstance(error, (ValidationError, RateLimitError)): level = 'WARNING' else: level = 'ERROR' client = get_client() client.update_current_span( level=level, status_message=str(error), metadata={'recoverable': isinstance(error, RetryableError)}, )

Visualizing Traces

Trace Timeline

Flamegraph

Best Practices

1. Meaningful Names

typescript
// Good startObservation('generation', { name: 'llm-inference' }) startObservation('span', { name: 'database-query' }) startObservation('span', { name: 'vector-search' }) // Avoid startObservation('span', { name: 'step1' }) startObservation('span', { name: 'process' }) startObservation('span', { name: 'func' })

2. Rich Metadata

python
client = get_client() client.update_current_span( output=response, metadata={ 'model': 'gpt-4', 'tokens': 350, 'cost': 0.0105, 'confidence': 0.95, 'cache_hit': False, 'retries': 0, }, )

3. Proper Error Handling

Always record errors in traces:

typescript
catch (error) { updateActiveObservation({ level: 'ERROR', statusMessage: error.message, metadata: { stack: error.stack /* + relevant data */ }, }) throw error // Still throw after recording }

4. Smart Sampling

Balance coverage and cost:

python
# Sample strategically def should_trace(request): # Always trace errors if has_error(request): return True # Always trace slow requests if is_slow(request): return True # Sample by user tier if request.user_tier == 'enterprise': return random.random() < 0.5 # 50% else: return random.random() < 0.1 # 10%

Next Steps

Explore Integrations →

© 2026 ANTS Platform, Inc.Docs v1.0 · Last updated June 2026