monocle2ai/monocle

Monocle – Observability & Testing for GenAI Apps

What it is – Monocle is an open‑source Python SDK that adds OpenTelemetry‑compatible tracing to generative‑AI applications and agents. It automatically instruments popular frameworks (LangChain, LlamaIndex, Haystack, Google ADK, Claude CLI, etc.) and emits structured spans for prompts, LLM calls, tool invocations, vector‑store queries, and agent‑run metadata. A companion package, monocle-test-tools, lets you write pytest‑style tests that assert on those traces instead of just on output values.

Why it matters – GenAI agents are often treated as black boxes, making debugging, cost monitoring, compliance, and CI testing difficult. Monocle turns every step of an agent’s execution into a visible timeline that can be sent to any OTLP collector, stored as JSON, or visualized with the Okahu VS Code extension.

---

Key Features (as described in the README)

Feature	What you get
Metamodel	A JSON schema defining standard span attributes for agents, prompts, responses, tools, and vector operations.
Auto‑instrumentation adapters	Plug‑and‑play support for many GenAI stacks (LangChain, LlamaIndex, Haystack, Google ADK, Claude CLI, Codex CLI, Copilot, AWS Bedrock, Azure OpenAI, etc.) – no manual span creation needed.
OTLP‑compatible export	Traces can be sent to any OpenTelemetry collector, written to local files, or pushed to cloud back‑ends (AWS S3, Azure Blob, Okahu cloud, PostgreSQL, ClickHouse, …).
Scope API	Context‑manager / decorator helpers (monocle_trace_scope, monocle_trace_scope_method) to attach business attributes (user‑id, tenant‑id, session‑id) to every span in a block.
Built‑in obfuscation	Automatic redaction of secrets from spans; configurable via environment variables (MONOCLE_OBFUSCATE_ENV_VALUES, MONOCLE_OBFUSCATE_EXTRA_PATTERNS).
Zero‑code wrapper mode	A CLI‑style entry point (monocle_apptrace …) lets you trace scripts without modifying source – useful for Lambda layers or platform‑level injection.
monocle-test-tools	pytest extensions that let you assert on trace contents: which agents ran, which tools were called, token/cost limits, duration, and higher‑level evaluation facts (role adherence, knowledge retention, etc.).
IDE integration	The Okahu VS Code extension reads Monocle JSON files and shows a Gantt‑style timeline with token counts and error badges.
Linux Foundation governance	Project is hosted under the LF AI & Data umbrella and follows Apache‑2.0 licensing.

---

Typical Use Cases

1. Debugging complex agents – Visualize every LLM call and tool use to pinpoint why an answer went wrong.
2. Cost & compliance monitoring – Trace token usage and automatically redact secrets; enforce token‑budget limits in CI.
3. CI/CD testing of AI flows – Write monocle-test-tools tests that verify the correct sequence of agent and tool invocations, not just the final output.
4. Platform‑level observability – Infra teams can inject tracing via the wrapper mode, giving SREs a uniform view of all GenAI services.
5. Multi‑tenant tracing – Use the Scope API to tag spans with tenant_id or session_id for easy filtering in dashboards.

---

Getting Started (quick‑start from the README)

# Install the SDK
pip install monocle_apptrace

# One‑time telemetry setup (usually at app start)
python - <<'PY'
from monocle_apptrace import setup_monocle_telemetry
setup_monocle_telemetry(workflow_name="my_genai_app")
PY

Running the app now creates JSON trace files under ./monocle/ (e.g., monocle_trace_my_genai_app_<trace_id>_<ts>.json).

For testing:

pip install monocle_test_tools

Then write pytest functions using expected or MonocleValidator as shown in the README examples.

---

Ecosystem & Integrations

• Framework adapters – LangChain, LlamaIndex, Haystack, Google ADK, Claude CLI, OpenAI Codex CLI, GitHub Copilot, AWS Bedrock Agentcore, Azure AI Foundry, etc.
• Export destinations – stdout, local file, in‑memory, Azure Blob, AWS S3, Okahu cloud, generic OTEL collectors, Google Cloud Storage, PostgreSQL, ClickHouse, Paygentic.
• IDE – Okahu Trace Visualizer VS Code extension for interactive timeline view.
• Language support – Primary Python SDK; a TypeScript companion repo (monocle-typescript) is linked.

---

License

Apache‑2.0 (as indicated by the badge in the README).

---

Bottom line – Monocle provides a ready‑made, OpenTelemetry‑based observability layer for generative‑AI applications, turning opaque agent runs into structured, searchable traces and enabling test‑driven validation of AI behavior.