openlit/openlit
Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.
What it solves
OpenLIT provides a comprehensive platform for AI engineering that simplifies the development, monitoring, and optimization of Generative AI and LLM applications. It addresses the difficulty of tracking performance, managing costs, and ensuring the quality of AI outputs across different models and frameworks.
How it works
OpenLIT uses OpenTelemetry-native SDKs (available in Python, TypeScript, and Go) to collect traces and metrics from AI applications. This data is sent to an OpenTelemetry Collector and stored in ClickHouse, where it can be visualized in the OpenLIT dashboard. It auto-instruments over 50 LLM providers, AI frameworks, and vector databases with a single line of code.
Who it’s for
AI developers and engineers who need full-stack observability, automated evaluation, and prompt management to move their AI features from testing to production confidently.
Highlights
- OpenTelemetry-native: Follows Semantic Conventions for vendor-neutral observability.
- Automated Evaluations: Includes 11 built-in LLM-as-a-Judge evaluation types for detecting hallucinations, bias, and toxicity.
- Prompt Management: A dedicated Prompt Hub for versioning and organizing prompts.
- Rule Engine: Conditional logic to dynamically retrieve prompts and evaluation configs at runtime.
- Coding Agent Observability: A CLI that provides observability for local coding agents like Claude Code, Cursor, and Codex.
- Cost Tracking: Precise budgeting with custom pricing files for custom and fine-tuned models.