Accio-Lab/Dressage

What it solves

Dressage provides a scalable reinforcement learning (RL) training framework specifically for LLM agents that interact with real-world tools (such as code editors, shells, and APIs). It solves the difficulty of bridging the gap between policy rollouts in isolated sandboxes and the training process, allowing for full RL gradient flow regardless of whether the agent is a "whitebox" Python loop or a "blackbox" HTTP-based agent.

How it works

Dressage acts as a bridge between the training substrate (built on the slime framework) and the agent's execution environment. It uses a three-layer architecture:

• Paddocks: Manage interaction semantics, handling how the agent calls tools or interacts with an HTTP server.
• Sandboxes: Provide isolation for tool execution, supporting either local bubblewrap pools or remote E2B sandboxes.
• Inference Proxy: Sits between the agent and the inference engine (SGLang), recording every token, logprob, and loss mask at a granular level to ensure training data is accurate and free from retokenization drift (via a system called TITO).

Who it’s for

It is designed for AI researchers and developers training LLM agents to perform complex tasks in software engineering (SWE) or general tool-use environments where scalable RL and secure sandboxing are required.

Highlights

• Universal Agent Support: Works with both whitebox Python agents and blackbox agents like Claude Code, OpenClaw, and Codex.
• Token-Wise Precision: Uses Token-In-Token-Out (TITO) to avoid retokenization drift and supports MoE Routing Replay (R3).
• Segment-Aware Training: Expands trajectory segments into training samples to maximize learning from history compaction or tool-schema changes.
• Pluggable Isolation: Seamlessly switches between local bubblewrap and remote E2B sandboxes.
• Production-Ready Safety: Includes atomic trajectory logging and context overflow detection to prevent training data corruption.