traceloop/openllmetry-js

Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based on OpenTelemetry

What it solves

OpenLLMetry-JS provides observability for LLM applications built with JavaScript and TypeScript. It helps developers track and monitor the internal workings of their AI applications, making it easier to debug and optimize performance by providing a standardized way to capture telemetry data from LLM providers and vector databases.

How it works

Built on top of OpenTelemetry, the project provides a set of extensions and a simplified SDK. It instruments calls to various LLM providers (like OpenAI, Anthropic, and Bedrock) and vector databases (like Pinecone and Chroma), capturing the data flow. Because it adheres to OpenTelemetry standards, the captured traces can be exported to a wide variety of existing observability tools such as Datadog, Honeycomb, and New Relic.

Who it’s for

Developers using Node.js or Next.js who are building LLM-powered applications and need a professional observability stack to monitor their AI workflows.

Highlights

  • OpenTelemetry-based: Uses industry-standard OpenTelemetry under the hood for compatibility with existing tools.
  • Broad Integration: Supports a wide range of LLM providers (OpenAI, Anthropic, Cohere, Vertex AI, Bedrock) and vector databases (Pinecone, Chroma, Qdrant).
  • Framework Support: Integrates with popular AI frameworks like LangChain and LlamaIndex.
  • Easy Setup: Offers a simplified SDK that allows instrumentation with just a few lines of code.

Related

  • Project
    traceloop/openllmetry

    An open-source observability framework built on OpenTelemetry that provides complete tracing and monitoring for LLM applications, providers, and vector databases.

  • Project
    Arize-ai/openinference

    OpenInference provides a vendor‑neutral specification and a suite of Python instrumentation packages that add OpenTelemetry‑style tracing to LLM and AI applications, covering prompts, retrieval, tool calls, and model responses.

  • Project
    openlit/openlit

    An open-source agent harness engineering platform that uses OpenTelemetry to trace, evaluate, and guard AI agents and coding agents.

  • Project
    monocle2ai/monocle

    Monocle is an Apache‑2.0 Python SDK that adds OpenTelemetry‑compatible tracing to GenAI apps and agents. It auto‑instrumentates popular frameworks (LangChain, LlamaIndex, Haystack, Google ADK, Claude CLI, etc.), emits structured spans for prompts, LLM calls, tool use, and vector queries, and integrates with any OTLP collector or local JSON files. A companion `monocle-test-tools` package lets you write pytest‑style tests that assert on those traces (agent runs, tool calls, token/cost limits, evaluation facts). Features include a metamodel for standard span attributes, scope helpers for tagging spans, built‑in secret obfuscation, zero‑code wrapper mode, VS Code visualizer integration, and broad exporter support (file, stdout, S3, Azure Blob, Okahu cloud, PostgreSQL, ClickHouse). Ideal for debugging, cost monitoring, CI testing, and platform‑level observability of AI agents.

  • Project
    langfuse/langfuse-python

    A Python SDK for the Langfuse platform that provides observability, tracing, prompt management, and evaluation tools for LLM applications.