microsoft/debug-gym

What it solves

debug-gym provides a standardized, interactive environment for training and evaluating LLM-based agents that fix bugs in Python code repositories. It bridges the gap between simple code editing and real-world debugging by allowing agents to use actual debugging tools (like pdb) to investigate code behavior before proposing a fix.

How it works

The framework operates as a simulation environment following the Gymnasium paradigm. An agent interacts with a code repository through a set of tools, receives text-based observations of the environment state, and iteratively generates commands to gather information and eventually apply a patch.

• Environment & Tools: The RepoEnv manages the codebase and provides tools such as bash, view (file focus), eval (test execution), pdb (interactive debugging), grep, listdir, and edit.
• Terminals: To ensure isolation and scalability, it supports DockerTerminal and KubernetesTerminal backends for executing commands.
• Agents: It provides LLM-based agents (e.g., froggy_agent) that can be configured with custom system prompts using Jinja templates.
• Integration: It integrates with benchmarks like SWE-bench, SWE-smith, and R2E-Gym to provide standardized debugging tasks.

Who it’s for

• AI researchers developing autonomous software engineering agents.
• Developers building LLM-based tools for automated bug fixing and program analysis.
• Users wanting to evaluate the debugging capabilities of LLM agents using real-world Python repositories.

Highlights

• Interactive Debugging: Integrates pdb to allow agents to set breakpoints and inspect state during execution.
• Extensible Tooling: Users can easily create and register new tools to expand the agent's action and observation space.
• Isolated Execution: Supports Docker and Kubernetes for safe, scalable execution of agent-generated code.
• Customizable Prompting: Uses Jinja templates for flexible system prompt engineering.
• Human-in-the-loop: Includes a "human mode" for manual interaction with the environment to test tool-calling workflows.