monocle2ai/monocle

Monocle is a framework for tracing GenAI app code. This repo contains implementation of Monocle for GenAI apps written in Python.

What it solves

Monocle provides observability and testing for Generative AI applications and agents. It transforms "black-box" AI agents into transparent, step-by-step traces, allowing developers to see every model call, tool invocation, and vector lookup in a connected timeline. It also solves the problem of testing AI agents by allowing assertions on the actual execution traces (e.g., verifying which tools were called) rather than just checking the final output.

How it works

Monocle is a GenAI-specific observability layer built on OpenTelemetry. It uses a metamodel to standardize how entities like agents, prompts, and vector operations are mapped to span attributes. It provides instrumentation adapters for popular frameworks (like LangChain, LlamaIndex, and Haystack) to automatically create traces without manual wiring. These traces are emitted as OTLP-compatible spans, which can be sent to any standard observability backend or visualized using the Okahu VS Code extension.

Who it’s for

  • App developers who want to trace GenAI apps without writing bespoke OpenTelemetry code.
  • Platform and infrastructure engineers who prefer to inject tracing via wrappers or operators rather than refactoring application code.
  • Enterprises and SREs who require standardized, OTel-compliant traces and CI-friendly tests for AI pipelines.

Highlights

  • Native OpenTelemetry Support: Emits standard OTLP spans compatible with any APM or collector.
  • Auto-Instrumentation: Supports a wide array of agentic frameworks, LLM providers, and vector stores.
  • Trace-Based Testing: Includes monocle-test-tools to assert on agent behavior, tool usage, token costs, and duration via pytest.
  • Flexible Deployment: Offers both in-code initialization and zero-code wrapper/operator modes for environments like AWS Lambda.
  • AI Assistant Tracing: Provides specialized setup hooks for tracing Claude CLI, OpenAI Codex CLI, and GitHub Copilot.