victordibia/designing-multiagent-systems

Building LLM-Enabled Multi Agent Applications from Scratch

What it solves

This project provides a practical, fundamentals-first approach to building multi-agent AI systems. It addresses the complexity of moving from simple LLM prompts to sophisticated multi-agent architectures by providing a custom-built framework called PicoAgents, which is designed for transparency and teaching rather than abstraction.

How it works

The project implements a complete multi-agent framework from scratch, covering the entire lifecycle of agent development:

  • Agent Core: Implements reasoning loops, tool calling, memory, and streaming.
  • Orchestration: Provides multiple coordination strategies, including sequential (round-robin), LLM-driven speaker selection, and plan-based orchestration.
  • Workflows: A type-safe engine for building DAG-based execution paths with streaming observability.
  • Specialized Agents: Includes implementations for computer-use (browser automation) and software engineering agents.
  • Infrastructure: A unified model client interface supporting OpenAI, Azure, Anthropic, and local LLMs, along with a Web UI for auto-discovery, debugging, and evaluation.

Who it’s for

Developers and AI engineers who want to understand the internal mechanics of multi-agent systems and learn how to implement coordination patterns, evaluation frameworks, and production-ready agentic workflows without being locked into a specific third-party framework.

Highlights

  • PicoAgents Framework: A full-featured framework built from scratch to expose the inner workings of agent reasoning and orchestration.
  • Diverse Orchestration Patterns: Implements GroupChat, LLM-driven, and plan-based coordination.
  • Computer Use: Built-in support for multimodal reasoning and browser automation.
  • Comprehensive Tooling: Includes a Web UI for real-time streaming chat, a debug rail, and an MCP (Model Context Protocol) playground.
  • Evaluation Suite: Integrated tools for LLM-as-judge metrics and batch run evaluation.

Related

  • Project
  • Project
  • Project
  • Project
  • Project