traceroot-ai/traceroot

What it solves

TraceRoot provides an observability and self-improvement loop for AI agents. It addresses the difficulty of manually sifting through massive amounts of production traces to find failures, the pain of root-causing errors across hallucinations and tool calls, and the lack of a systematic way to improve agent performance over time.

How it works

TraceRoot uses an OpenTelemetry-compatible SDK to capture LLM calls, agent actions, and tool usage. It employs "Detectors" (LLM-as-judge evaluators) to automatically monitor traces for hallucinations, logic failures, and safety violations. When a failure is detected, the system triggers an agentic debugging process that connects to a sandbox with the production source code and GitHub history to identify the failing line of code and open a verified fix PR.

Who it’s for

Developers building AI agents using frameworks like LangChain, CrewAI, AutoGen, or Pydantic AI, who need to monitor production performance and automate the debugging and improvement cycle.

Highlights

• Agentic Debugging: Automatically identifies failing lines of code and correlates failures with GitHub commits and PRs to open fix PRs.
• LLM-as-Judge Detectors: Automatically screens traces for hallucinations, tool failures, and intent drift.
• Closed-Loop Improvement: Converts production failures into golden datasets for offline evaluation and verification of fixes.
• Broad Integration: Supports a wide range of agent frameworks (e.g., LlamaIndex, DSPy, Vercel AI SDK) and model providers (OpenAI, Anthropic, Gemini).
• Open Source: Fully open source with BYOK (Bring Your Own Key) support for any model provider.