LangChain Integration
Integrate ANTS Platform with LangChain for automatic tracing of your LLM applications.
Overview
LangChain is the most popular framework for building LLM applications. ANTS Platform provides seamless integration through callbacks and auto-instrumentation.
Features
- Automatic tracing of all LangChain operations
- Chain execution tracking (Sequential, Router, etc.)
- LLM calls monitoring with token usage and costs
- Tool usage tracking
- Memory and context tracking
- Multi-agent workflows
- Retrieval operations (vector stores, documents)
Installation
bash
npm install ants-platform @antsplatform/langchain @langchain/openai @langchain/core
Quick Start
Callback Handler
The integration works through a LangChain callback handler. Attach it per-invocation and every chain, LLM, and tool call is traced.
typescript
import { CallbackHandler } from '@antsplatform/langchain'
import { ChatOpenAI } from '@langchain/openai'
// ANTS_PLATFORM_PUBLIC_KEY / ANTS_PLATFORM_SECRET_KEY are read from the env
const handler = new CallbackHandler()
const llm = new ChatOpenAI({ model: 'gpt-4' })
const response = await llm.invoke('What is AI?', {
callbacks: [handler],
})
// Automatically traced in ANTS Platform.
Tracing requires Node.js 20+. Set
ANTS_PLATFORM_PUBLIC_KEY,ANTS_PLATFORM_SECRET_KEY, andANTS_PLATFORM_BASE_URL=https://api.antsplatform.aiin your environment.
Adding Session, User, and Metadata
For richer traces, pass sessionId, userId, tags, and traceMetadata when constructing the handler:
typescript
import { CallbackHandler } from '@antsplatform/langchain'
import { ChatOpenAI } from '@langchain/openai'
const handler = new CallbackHandler({
sessionId: 'customer-support',
userId: 'user_123',
tags: ['production'],
traceMetadata: {
environment: 'production',
version: '1.0.0',
},
})
const llm = new ChatOpenAI({ model: 'gpt-4' })
const response = await llm.invoke('Help me with my order', {
callbacks: [handler],
})
Advanced Usage
Chains
Track complex chains:
python
from langchain.chains import ConversationalRetrievalChain
from langchain_openai import ChatOpenAI
from langchain_community.vectorstores import Pinecone
from ants_platform.langchain import CallbackHandler
# Set up components
llm = ChatOpenAI()
vectorstore = Pinecone.from_existing_index('my-index')
# Create the ANTS Platform callback handler
handler = CallbackHandler()
chain = ConversationalRetrievalChain.from_llm(
llm=llm,
retriever=vectorstore.as_retriever(),
)
# Run chain - fully traced!
result = chain.invoke(
{"question": "What are the product features?", "chat_history": []},
config={"callbacks": [handler]},
)
# ANTS Platform captures:
# - Question asked
# - Documents retrieved
# - LLM prompt and response
# - Total tokens and cost
# - Execution time per step
Agents
Monitor LangChain agents:
typescript
const llm = new ChatOpenAI({ temperature: 0 })
const tools = [new SerpAPI()]
const handler = new CallbackHandler({
sessionId: 'research-agent',
traceMetadata: { agentType: 'zero-shot' },
})
const agent = ZeroShotAgent.fromLLMAndTools(llm, tools)
const executor = new AgentExecutor({ agent, tools })
const result = await executor.invoke(
{ input: "What's the weather in San Francisco?" },
{ callbacks: [handler] },
)
// ANTS Platform tracks:
// - Agent planning steps
// - Tool calls (SerpAPI)
// - LLM reasoning
// - Final answer
Memory
Track conversation memory:
python
from langchain.memory import ConversationBufferMemory
from langchain.chains import ConversationChain
memory = ConversationBufferMemory()
chain = ConversationChain(llm=llm, memory=memory)
# Each conversation turn is traced
config = {"callbacks": [handler]}
chain.invoke({"input": "Hi, I'm looking for a laptop"}, config=config)
chain.invoke({"input": "What are some good options under $1000?"}, config=config)
chain.invoke({"input": "Tell me more about the first one"}, config=config)
# ANTS Platform shows:
# - Full conversation history
# - Memory usage
# - Context provided to LLM
LangGraph Integration
For LangGraph (stateful multi-actor applications):
python
from langgraph.graph import StateGraph
from ants_platform.langchain import CallbackHandler
# Define your graph
workflow = StateGraph(AgentState)
# Add nodes
workflow.add_node("researcher", research_node)
workflow.add_node("writer", write_node)
workflow.add_node("editor", edit_node)
# Add edges
workflow.add_edge("researcher", "writer")
workflow.add_edge("writer", "editor")
# Compile with ANTS Platform tracing
app = workflow.compile()
handler = CallbackHandler()
# Run with tracing
result = app.invoke(
{"topic": "AI agents"},
config={"callbacks": [handler]}
)
# ANTS Platform visualizes:
# - Complete graph execution
# - Each node's input/output
# - State transitions
# - Timing per node
What Gets Tracked
LLM Calls
json
{
"type": "generation",
"model": "gpt-4",
"input": "What is AI?",
"output": "AI is...",
"usage": {
"input": 15,
"output": 150,
"total": 165
},
"latencyMs": 1250
}
Chain Execution
json
{
"type": "span",
"name": "ConversationalRetrievalChain",
"input": "What are the features?",
"output": "The features include...",
"observations": [
{ "name": "retrieval", "docs": 5, "latencyMs": 200 },
{ "name": "llm", "totalTokens": 350, "latencyMs": 1500 }
]
}
Tool Usage
json
{
"type": "span",
"name": "search",
"input": "weather in SF",
"output": "68F and sunny",
"latencyMs": 450
}
Best Practices
1. Group Traces with a Session
python
handler = CallbackHandler()
result = chain.invoke(
{"input": query},
config={
"callbacks": [handler],
"metadata": {
"session_id": "customer-support-bot",
"channel": "web",
"version": "2.1.0",
},
},
)
2. Add Rich Metadata
typescript
const handler = new CallbackHandler({
userId: 'user_123',
sessionId: 'session_abc',
traceMetadata: {
customerTier: 'enterprise',
feature: 'qa-bot',
},
})
3. Handle Errors
python
try:
result = chain.invoke({"input": query}, config={"callbacks": [handler]})
except Exception as error:
# Error is automatically captured by ANTS Platform
logger.error(f"Chain failed: {error}")
4. Sample Appropriately
typescript
// Only trace in production for high-value requests
const shouldTrace = (request) => {
if (process.env.NODE_ENV !== 'production') return false
if (request.userTier === 'enterprise') return true
return Math.random() < 0.1
}
const callbacks = shouldTrace(request) ? [new CallbackHandler()] : []
const response = await llm.invoke(prompt, { callbacks })
Example Projects
RAG System
Complete retrieval-augmented generation:
python
from langchain.chains import RetrievalQA
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_community.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
from ants_platform.langchain import CallbackHandler
# Load and process documents
loader = TextLoader('docs.txt')
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000)
docs = text_splitter.split_documents(documents)
# Create vector store
embeddings = OpenAIEmbeddings()
vectorstore = FAISS.from_documents(docs, embeddings)
# Create the ANTS Platform callback handler
handler = CallbackHandler()
qa = RetrievalQA.from_chain_type(
llm=ChatOpenAI(),
retriever=vectorstore.as_retriever(),
)
# Query with full observability
answer = qa.invoke(
{"query": "What is mentioned about pricing?"},
config={"callbacks": [handler]},
)
Multi-Agent System
Collaborative agents:
typescript
const researchHandler = new CallbackHandler({ sessionId: 'researcher-agent' })
const writerHandler = new CallbackHandler({ sessionId: 'writer-agent' })
const researcher = new AgentExecutor({
agent: researchAgent,
tools: researchTools,
})
const writer = new AgentExecutor({
agent: writerAgent,
tools: writerTools,
})
// Coordinate agents
const research = await researcher.invoke(
{ input: topic },
{ callbacks: [researchHandler] },
)
const article = await writer.invoke(
{ input: `Write about: ${research.output}` },
{ callbacks: [writerHandler] },
)
// ANTS Platform shows both agents' work
Troubleshooting
Callbacks Not Working
python
# Make sure callbacks are passed correctly via config
chain.invoke({"input": query}, config={"callbacks": [handler]}) # Correct
# Not this - constructing the chain without passing the handler at invocation
chain.invoke({"input": query}) # Won't be traced
Missing Token Counts
typescript
// Ensure you're using a model that reports usage
const llm = new ChatOpenAI({
model: 'gpt-4', // Reports usage
})
await llm.invoke(prompt, { callbacks: [handler] })
High Credit Usage
python
# Use sampling to reduce costs
callbacks = [handler] if random.random() < 0.1 else [] # 10% sampling
chain.invoke({"input": query}, config={"callbacks": callbacks})
Next Steps
- Python SDK - Direct Python integration
- LlamaIndex - Another popular framework
- Guides - End-to-end tutorials