oxbshw/LLM-Agents-Ecosystem-Handbook

One-stop handbook for building, deploying, and understanding LLM agents with 60+ skeletons, tutorials, ecosystem guides, and evaluation tools.

What it solves

This project provides a comprehensive, production-oriented operating manual for building and shipping modern LLM agent systems. It moves beyond simple prompting by treating agents as full systems comprising identity, memory, skills, tools, and safety guardrails.

How it works

The handbook is organized into a modular stack that covers the entire agent lifecycle:

  • Provider Layer: An LLMProvider abstraction and ProviderRouter that support 24+ LLM providers (frontier, fast inference, marketplaces, enterprise, and local runtimes) to allow for easy switching and fallback chains.
  • Agent OS Concept: A framework for defining agent identity, memory (durable state), and skills (repeatable, multi-step workflows).
  • MCP Integration: Guidance and templates for using the Model Context Protocol (MCP) to standardize how agents access external tools and context.
  • Production Readiness: A suite of design documents, ADR (Architecture Decision Record) guides, safety checklists, and evaluation frameworks to ensure agents are secure and measurable.
  • Coding Agent Harness: Specific workflows and prompts optimized for tools like Claude Code, Cursor, and Aider.

Who it’s for

Developers and architects building production-grade AI agents who need a standardized way to organize their agent's identity, memory, and tool-use patterns, or those comparing AI agent frameworks (e.g., LangGraph, CrewAI, Pydantic AI).

Highlights

  • Multi-Provider Support: Unified interface for 24+ LLM providers with a built-in router for task-based routing.
  • Extensive Templates: Copy-paste ready templates for system prompts, agent identity (SOUL.md), memory indices (MEMORY.md), and design docs.
  • Curation of the Ecosystem: A detailed comparison matrix of major agent frameworks and a catalog of 100+ agent skeletons.
  • Risk-Tiered Safety: A structured approach to tool access based on risk levels (Low to Critical) with required human approval gates.
  • Evaluation Framework: Playbooks for tracing, cost/latency analysis, and specific eval datasets for tool-calling and memory.