traceloop/openllmetry

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

What it solves

OpenLLMetry provides open-source observability for LLM applications. It solves the problem of tracking and monitoring the complex interactions within AI apps, allowing developers to see exactly what is happening inside their LLM calls, vector database queries, and framework orchestrations.

How it works

Built on top of OpenTelemetry, OpenLLMetry provides a set of extensions and a dedicated SDK that instrument your code. By adding a few lines of initialization, it automatically captures traces from LLM providers (like OpenAI and Anthropic), vector databases (like Pinecone and Chroma), and AI frameworks (like LangChain and LlamaIndex). Because it follows OpenTelemetry standards, the captured data can be exported to a wide variety of existing observability backends such as Datadog, Honeycomb, and Grafana.

Who it’s for

Developers building LLM-powered applications who need to debug, monitor, and optimize their AI workflows without being locked into a proprietary observability tool.

Highlights

  • OpenTelemetry Native: Uses standard OpenTelemetry data, ensuring compatibility with any OTEL-compliant backend.
  • Broad Integration Support: Instruments a vast array of LLM providers, vector databases, and AI frameworks.
  • Hassle-Free Setup: Offers a SDK that allows for near-instant instrumentation with a single Traceloop.init() call.
  • Extensible: Can be used as a full SDK or as standalone instrumentations for those who already have an OpenTelemetry setup.

Related

  • Project
    traceloop/openllmetry-js

    An open-source observability library for JavaScript/TypeScript LLM applications that uses OpenTelemetry to trace calls to LLM providers and vector databases.

  • Project
    openlit/openlit

    An open-source agent harness engineering platform that uses OpenTelemetry to trace, evaluate, and guard AI agents and coding agents.

  • Project
    Arize-ai/openinference

    OpenInference provides a vendor‑neutral specification and a suite of Python instrumentation packages that add OpenTelemetry‑style tracing to LLM and AI applications, covering prompts, retrieval, tool calls, and model responses.

  • Project
    Arize-ai/phoenix

    Arize Phoenix is an open‑source AI observability platform for LLM apps, offering tracing, evaluation, dataset/experiment tracking, prompt playground, and an AI‑assistant (PXI). It installs via pip/conda, runs locally or in Docker/Kubernetes, and integrates with major LLM frameworks and providers.

  • Project
    monocle2ai/monocle

    Monocle is an Apache‑2.0 Python SDK that adds OpenTelemetry‑compatible tracing to GenAI apps and agents. It auto‑instrumentates popular frameworks (LangChain, LlamaIndex, Haystack, Google ADK, Claude CLI, etc.), emits structured spans for prompts, LLM calls, tool use, and vector queries, and integrates with any OTLP collector or local JSON files. A companion `monocle-test-tools` package lets you write pytest‑style tests that assert on those traces (agent runs, tool calls, token/cost limits, evaluation facts). Features include a metamodel for standard span attributes, scope helpers for tagging spans, built‑in secret obfuscation, zero‑code wrapper mode, VS Code visualizer integration, and broad exporter support (file, stdout, S3, Azure Blob, Okahu cloud, PostgreSQL, ClickHouse). Ideal for debugging, cost monitoring, CI testing, and platform‑level observability of AI agents.