Ajay150313/agentsre

SRE reliability instrumentation for agentic AI — DQR, TIE, HER, AQD

What it solves

Traditional observability tools (like Datadog or CloudWatch) monitor infrastructure health (HTTP 200s, latency, uptime) but fail to detect "semantic failures"—situations where an AI agent is technically running but making wrong decisions, escalating too many tasks to humans, or becoming inefficient. agentsre provides Site Reliability Engineering (SRE) instrumentation specifically for the semantic layer of agentic AI to prevent production outages caused by "agent sprawl" and behavioral drift.

How it works

The library implements a set of specialized Service Level Indicators (SLIs) and governance tools to monitor agent behavior:

  • Semantic SLIs: It tracks Decision Quality Rate (DQR), Tool Invocation Efficiency (TIE), Human Escalation Rate (HER), and Approval Queue Depth Drift (AQDD) to identify degradation before it causes a system failure.
  • A2A Validation: An Agent-to-Agent (A2A) semantic boundary validator ensures that output from one agent is semantically correct before it is passed to the next agent in a chain.
  • Circuit Breakers: A semantic circuit breaker can automatically trip and route requests to a degraded mode handler if the validation success rate drops below a specific threshold.
  • Fleet Governance: An inventory system tracks model versions, SLO ownership, and deprecation dates, while a framework canary prevents upgrading agent frameworks (e.g., LangChain) if behavioral drift is detected.
  • Autonomy Constraints: A "constraint ladder" can automatically reduce an agent's autonomy (e.g., moving from autonomous writes to human-approval-only) based on SLI breaches.

Who it’s for

SREs and AI engineers deploying multi-model, multi-framework agentic AI systems in production who need to move beyond basic infrastructure monitoring to behavioral reliability.

Highlights

  • Four Core Semantic SLIs: Specifically designed to catch failures that standard health checks miss.
  • AWS Integration: Built-in publisher for AWS CloudWatch custom metrics.
  • Curbing Agent Sprawl: Tools for fleet inventory and framework upgrade canaries.
  • Semantic Guardrails: A2A validation and circuit breakers that operate on decision quality rather than HTTP status codes.
  • Cost Tracking: A module for real-time cost tracking per agent and per task.

Related

  • Project
    sophgo/LLM-TPU

    LLM‑TPU is an open‑source SOPHGO project that compiles HuggingFace LLM and vision‑language models into SOPHGO TPU‑compatible bmodels. It supports one‑click compilation, a large catalog of quantised models, dynamic/KV‑cache inference, multi‑chip scaling, and provides ready Python/C++ demos for BM1684X, BM1688 and CV186X chips.

  • Project
    skilld-dev/skilld

    A curated registry and CLI for discovering, installing, and managing human-authored skills for AI agents across multiple platforms.

  • Project
    AvdLee/Core-Data-Agent-Skill

    A specialized knowledge set for AI coding agents that provides expert guidance on Core Data persistence, performance optimization, and schema migration for Apple platforms.

  • Project
    Dogacel/auto-gpu-kernel

    An autonomous GPU-kernel discovery and optimizer that uses a robust benchmarking pipeline to automatically find and implement high-performance GPU kernels.