Arize-ai/phoenix

What it solves

Arize Phoenix is an open-source AI observability platform that helps developers troubleshoot, evaluate, and experiment with LLM applications. It addresses the challenge of understanding what happens inside complex AI workflows by providing visibility into runtime traces, performance benchmarking, and prompt optimization.

How it works

Phoenix uses OpenTelemetry-based instrumentation (via the OpenInference project) to capture runtime traces of LLM applications. It is vendor and language agnostic, integrating with popular frameworks like LangGraph, LlamaIndex, and CrewAI, as well as LLM providers such as OpenAI, Anthropic, and Google GenAI. The platform can be run locally, in a container, or in the cloud.

Who it’s for

AI engineers and developers building LLM-powered applications who need to debug their traces, benchmark performance using LLM-based evaluations, and systematically iterate on prompts and models.

Highlights

• Tracing: Capture and visualize LLM application runtime using OpenTelemetry.
• Evaluation: Use LLMs to benchmark performance through response and retrieval evaluations.
• Datasets & Experiments: Create versioned datasets to track changes to prompts, LLMs, and retrieval methods.
• Playground: Optimize prompts, compare models, and replay traced LLM calls.
• PXI (Phoenix Intelligence): An integrated AI engineering agent for debugging and prompt iteration.
• Remote MCP Server: Connect coding agents like Claude Code and Cursor directly to Phoenix to query traces and datasets.