Opik
A self-hosted LLM observability platform for tracing agents, evaluating answers and monitoring production, with Apache 2.0 code and a hosted service.
Opik is an LLM observability and evaluation platform for teams building AI agents and retrieval-based applications. It helps developers inspect failures and compare behavior across development, testing and production.
- Agent tracing: Inspect model calls, retrieval steps, tool executions and conversations in connected trace trees, and annotate steps with feedback.
- Evaluation and tests: Compare results across datasets and experiments using code-based metrics or LLM judges for hallucinations, answer relevance and task completion, with PyTest integration for development checks.
- Prompt experiments: Test prompts and models in a playground, manage prompts, and use the Agent Optimizer to improve prompts and agent behavior.
- Production monitoring: Track feedback scores and token usage over time, and apply online evaluation rules to identify interactions that fail your criteria.
Opik works across agent frameworks, with integrations including Google ADK, Autogen, AG2 and Flowise AI. It also integrates with Anthropic and Amazon Bedrock.
The open source core observability and evaluation features are free to self-host under Apache 2.0. You can run that deployment in your own environment; Comet also offers a hosted service.
Logotyper
