LangWatch
A self-hosted AI agent testing and LLM evaluation platform with text and voice simulations, OpenTelemetry tracing and a managed cloud option.
LangWatch helps AI teams test agent behavior before release and investigate production failures. Teams can reproduce a failed conversation and check how an agent handles it again.
- Conversation simulations: Run multi-turn text and voice scenarios locally or in CI, and turn production traces into repeatable tests.
- Adversarial testing: Probe jailbreaks, policy violations and unsafe tool calls through an agent's API or internals, with support for mocking tools and MCP servers.
- LLM evaluations: Score responses, full conversations and images with an LLM judge, custom code or a workflow, and compare outputs to assess prompts or models.
- Production traces: Inspect tool calls, token use, cost and latency through OpenTelemetry, with plain-language search and conversation grouping by topic.
- AI usage management: Manage prompts and route requests through an OpenAI- and Anthropic-compatible gateway with virtual keys and budgets, alongside tools for tracking company AI usage.
Coding assistant integrations include Claude Code, Codex, Cursor and GitHub Copilot. Evaluations can run in a Jupyter notebook or the team UI, as well as on live production traffic.
LangWatch is open source under Apache 2.0. You can self-host it with Docker or Kubernetes, including in your own VPC, or use its managed cloud service. Hybrid deployment keeps the data plane on your infrastructure while LangWatch hosts the control plane.
