Arize-ai/openinference
OpenTelemetry Instrumentation for AI Observability
What is OpenInference?
OpenInference is a standard and a set of ready‑to‑use plugins that let you add OpenTelemetry‑style tracing to any AI/LLM application. It defines a specification (a collection of markdown files) that describes how to record the flow of prompts, model calls, retrieval from vector stores, tool use, etc. The spec is format‑agnostic, so you can emit the data as JSON, protobuf, DataFrames, or any other transport.
Why it matters
- Observability for LLM apps – just as OpenTelemetry gives you insight into micro‑service latency and errors, OpenInference gives you visibility into the hidden steps of generative‑AI pipelines (prompt construction, retrieval, tool calls, model responses).
- Vendor‑neutral – the spec works with any OpenTelemetry‑compatible backend (e.g., Jaeger, Prometheus, Grafana) and is already baked into Arize’s own products (Phoenix, AX) but can be used elsewhere.
- Broad language support – the repository ships a large collection of Python instrumentation packages for the most popular LLM frameworks and services (OpenAI, Anthropic, LangChain, LlamaIndex, Bedrock, VertexAI, etc.).
Core components
| Component | What it does | How you use it |
|---|---|---|
Specification (/spec) |
Defines semantic conventions for tracing LLM invocations, retrieval, tool usage, and more. | Follow the markdown spec when emitting trace data; libraries can generate it for you. |
Semantic‑conventions library (openinference-semantic-conventions) |
Provides the constant names and attribute definitions that trace payloads should contain. | Import the constants in your instrumentation code to ensure compliance. |
Instrumentation utilities (openinference-instrumentation) |
Helper functions, decorators, and configuration objects that make it easy to wrap existing code. | Add a decorator or call a helper around your LLM client calls. |
Framework‑specific plugins (e.g., openinference-instrumentation-openai, ‑langchain, ‑llama-index, ‑anthropic, …) |
Auto‑instrument popular SDKs so that every request/response is traced without manual boilerplate. | Install the package that matches the SDK you use; it patches the SDK at import time. |
Getting started (Python example)
# Install the core conventions and the plugin for the SDK you use
pip install openinference-semantic-conventions openinference-instrumentation-openai
from openinference.instrumentation.openai import OpenAIInstrumentor
from openai import OpenAI
# Enable automatic tracing
OpenAIInstrumentor().instrument()
client = OpenAI()
# This call will now produce OpenTelemetry spans that follow the OpenInference spec
response = client.chat.completions.create(model="gpt-4", messages=[{"role": "user", "content": "Hello"}])
The generated spans can be sent to any OpenTelemetry collector (e.g., Jaeger, Zipkin) and visualised in dashboards, giving you latency, error rates, token usage, and a full view of the LLM workflow.
Who should use it?
- ML engineers and data scientists building production LLM services who need monitoring and debugging.
- DevOps / SRE teams that already run OpenTelemetry and want to extend observability to AI workloads.
- Product teams that want to audit usage, detect anomalies, or measure performance across different model providers.
TL;DR – OpenInference is a community‑driven spec plus a rich ecosystem of Python plugins that bring OpenTelemetry‑style tracing to generative‑AI applications, making LLM pipelines observable, debuggable, and portable across monitoring back‑ends.
Related
- Project
- Project
- Project
- Project
- Project