Arize-ai/phoenix

AI Observability & Evaluation

What is Arize Phoenix?

Arize Phoenix is an open‑source AI observability platform. It lets you trace, evaluate, and debug large‑language‑model (LLM) applications, and it provides a UI for experimenting with prompts, datasets, and model versions.

Core capabilities

Feature What it does How you use it
Tracing Collects runtime data from LLM calls (inputs, outputs, latency, tokens, etc.) via OpenTelemetry‑based instrumentation. Run px setup (or the @arizeai/phoenix-cli) – it auto‑detects your framework/provider and injects the OpenInference instrumentation.
Evaluation Runs automated LLM‑based benchmarks (response quality, retrieval relevance, etc.) on stored traces or custom datasets. Use the arize-phoenix-evals / @arizeai/phoenix-evals library or the UI’s evaluation tab.
Datasets & Experiments Version‑controlled collections of examples that can be used for prompt engineering, fine‑tuning, or evaluation. Create datasets through the UI or via the client libraries; launch experiments to compare prompt or model changes.
Playground & Prompt Management Interactive UI to tweak prompts, compare model outputs, tag versions, and replay traced calls. Open the Phoenix web UI, select a trace, edit the prompt, and re‑run it instantly.
PXI (Phoenix Intelligence) An AI‑driven “coding agent” that can answer questions about your traces, suggest prompt changes, or navigate the product. Ask PXI inside the UI or via the MCP endpoint.
Remote MCP Server A LangChain‑style “model‑code‑protocol” endpoint (/mcp) that lets tools like Claude Code, Cursor, or other MCP clients query Phoenix data programmatically. Call the /mcp HTTP endpoint from a supported client.

Who might need it?

  • LLM developers who want to see exactly what their app is sending to a model and what it returns.
  • Prompt engineers looking for a systematic way to version, test, and compare prompts.
  • ML Ops teams that need a self‑hosted observability stack for LLM‑powered services (Phoenix can run locally, in Docker, or on Kubernetes).
  • Researchers who want reproducible datasets and experiment tracking for RAG or fine‑tuning pipelines.

Installation & deployment

  • Python/conda: pip install arize-phoenix (or conda install -c conda-forge arize-phoenix).
  • Run locally: phoenix serve starts the UI and API.
  • Container: Docker image arizephoenix/phoenix on Docker Hub; Helm chart available for Kubernetes.
  • One‑click cloud: Buttons for Railway, Render, Google Cloud, Azure, and AWS CloudFormation are provided.

Ecosystem & integrations

  • Frameworks: OpenAI Agents SDK, Claude Agent SDK, LangChain, Vercel AI SDK, Mastra, CrewAI, LlamaIndex, DSPy, etc.
  • LLM providers: OpenAI, Anthropic, Google GenAI, AWS Bedrock, OpenRouter, LiteLLM, and more.
  • Instrumentation: Built on the OpenInference project for OpenTelemetry‑compatible tracing.
  • Client libraries: Light‑weight Python (arize-phoenix-client, arize-phoenix-otel, arize-phoenix-evals) and TypeScript (@arizeai/phoenix-client, @arizeai/phoenix-otel, @arizeai/phoenix-evals).

License & community

  • The core platform is open source (MIT‑style, see the repo). A managed SaaS version called Arize AX is also offered.
  • Community channels: Slack, Bluesky, X (Twitter), LinkedIn, and a public GitHub issue tracker.

Bottom line

Arize Phoenix gives you a full‑stack observability suite for LLM applications—tracing, evaluation, prompt versioning, and an AI‑assistant—all self‑hostable and compatible with the major LLM toolkits.

Related

  • Project
    langfuse/langfuse

    An open-source LLM engineering platform for observability, prompt management, and evaluation to help teams develop and debug AI applications.

  • Dispatch
    Tracing and Evaluating smolagents with Arize Phoenix

    Hugging Face demonstrates how to use Arize Phoenix with smolagents to implement real-time tracing and LLM-as-a-judge evaluations for agentic workflows.

  • Project
    comet-ml/opik

    An open-source LLM observability and evaluation platform for AI agent tracing, prompt management, and production monitoring.

  • Project
    lmnr-ai/lmnr

    Laminar is an open-source observability platform for AI agents that provides automated tracing, evaluation tools, and real-time monitoring to help developers debug and optimize agent performance.

  • Project
    Helicone/helicone

    An AI Gateway and LLM observability platform that provides unified API access to 100+ models, request tracing, and prompt management for AI engineers.