Arize-ai/openinference
OpenTelemetry Instrumentation for AI Observability
What it solves
OpenInference provides a standardized way to trace and observe AI applications. It solves the problem of fragmented observability for LLM-based apps, where developers need to gain insight into LLM invocations, retrieval from vector stores, and the use of external tools (like search engines or APIs) across different frameworks.
How it works
It acts as a set of conventions and plugins that complement OpenTelemetry. By providing a specification for how AI-related data should be recorded, it allows developers to use a wide variety of instrumentations for popular ML SDKs and frameworks (such as OpenAI, LangChain, LlamaIndex, and others). These instrumentations capture the application context and can be used with any OpenTelemetry-compatible backend, including Arize Phoenix and Arize AX.
Who it’s for
It is designed for developers building AI agents and LLM applications who need a consistent, transport-agnostic way to monitor and trace the execution flow of their systems.
Highlights
- Standardized Specification: A transport- and file-format agnostic specification for tracing LLM apps.
- Broad Framework Support: Includes instrumentations for a vast array of tools including OpenAI, Anthropic, LangChain, LlamaIndex, CrewAI, and DSPy.
- OpenTelemetry Integration: Built to be complementary to the OpenTelemetry standard, ensuring compatibility with a wide range of backends.
- Span Processors: Provides utilities to normalize and convert data from other instrumentation libraries like OpenLIT and OpenLLMetry.