Arize Phoenix

Arize Phoenix helps developers investigate failures in AI agents and LLM applications and compare proposed fixes. It connects recorded application activity with evaluations and repeatable experiments.

• Trace inspection: Examine prompts, retrieved context, tool calls and model outputs to find where a response went wrong.
• Quality evaluation: Add human annotations or use LLM judges to score responses and retrieval, alongside cost and latency.
• Repeatable experiments: Create versioned datasets from traces and compare changes to prompts, models or retrieval under the same conditions.
• Prompt testing: Compare models, replay recorded LLM calls and track prompt revisions and tags through the playground and prompt management tools.
• Agent-assisted investigation: The built-in PXI agent helps investigate traces and revise prompts, while an MCP endpoint lets Claude Code and Cursor query traces, datasets and experiments.

Phoenix runs locally on a laptop or on your infrastructure through Docker or Kubernetes. Self-hosting keeps stored traces in your environment; Arize also offers a hosted cloud platform. Phoenix uses the source-available Elastic License 2.0 (ELv2).

OpenTelemetry-based tracing works across languages and frameworks. Integrations include Ollama for local LLM applications, OpenAI and Anthropic, plus LangGraph, LlamaIndex and Vercel AI SDK.