TruLens

TruLens helps developers assess AI agents and retrieval-augmented generation (RAG) apps, then trace poor results to the steps that caused them. It connects evaluation scores with execution traces so teams can compare changes in quality, latency and cost.

• Step-level tracing records inputs, outputs, latency, token use and cost, including MCP tool arguments and results.
• Agent and RAG evaluation checks tool choice, argument validity, plan quality and redundant work, alongside retrieval relevance and whether answers have support in retrieved material.
• Custom scoring lets you define Python metrics or LLM judge prompts, with domain-specific rubrics and examples; judges explain their scores.
• Version comparison shows quality, latency and cost across application versions, with evaluations that run as traces arrive or against an existing dataset.

TruLens is open source under the MIT license. It works with custom Python apps, LangChain, LangGraph and LlamaIndex, and can store results in a database you run. OpenTelemetry traces can go to Jaeger, Grafana Tempo or another OTLP-compatible backend.

Judge integrations include HuggingFace and LangChain models, plus OpenAI, Google Gemini, AWS Bedrock and Snowflake Cortex. Evaluations using cloud judges send evaluation requests to the selected provider.