openlit: an OpenTelemetry-native observability and evaluation platform for AI engineering

What it solves

OpenLIT is an open-source platform designed to simplify the AI engineering workflow for Generative AI and LLMs. It addresses the challenges of monitoring performance, managing prompts, securing API keys, and evaluating model outputs in a vendor-neutral way.

How it works

OpenLIT uses OpenTelemetry-native SDKs (available in Python, TypeScript, and Go) to collect traces and metrics from LLMs, vector databases, and GPUs. This data is sent to an OpenTelemetry Collector and stored in ClickHouse, where it can be visualized via the OpenLIT UI. It also provides a CLI for monitoring local coding agents like Cursor and Claude Code by installing vendor hooks that emit traces.

Who it’s for

AI engineers and developers building LLM-powered applications who need full-stack observability, automated evaluation, and centralized management of prompts and secrets.

Highlights

• OpenTelemetry-native: Follows semantic conventions for vendor-neutral observability.
• Automated Evaluations: Includes 11 built-in evaluation types (e.g., hallucination, bias, toxicity) using LLM-as-a-Judge.
• Prompt Management: Centralized versioning and organization of prompts via Prompt Hub.
• Rule Engine: Conditional logic to dynamically retrieve contexts, prompts, and evaluation configurations.
• Broad Integration: Auto-instruments over 50 LLM providers, AI frameworks (like LangChain and LlamaIndex), and vector databases.
• Coding Agent Observability: Dedicated CLI to monitor sessions and tool calls for local AI coding tools.