hamishivi/tmax

What it solves

Tmax provides a comprehensive framework for creating powerful AI agents capable of using a computer terminal. It streamlines the entire lifecycle of building these agents, including generating synthetic training data, training the models, and evaluating their performance on complex terminal tasks.

How it works

The project is organized into four primary stages:

1. Data Generation: Uses a scalable pipeline to synthesize diverse, difficulty-aware terminal tasks. These tasks are packaged as self-contained Docker or Apptainer environments with programmatic verifiers to check if the agent solved the task.
2. Agent Implementation: Employs the "Vanillux2Agent," a LiteLLM-based agent that uses specific prompt harnesses, a bash tool schema, and format-error recovery to execute commands in active environments.
3. Training: Utilizes a fork of open-instruct to perform Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPPO) Reinforcement Learning (RL) to optimize the models.
4. Evaluation: Uses vLLM for model serving and a Beaker pipeline to test models against datasets like Terminal-Bench and SWE-bench.

Who it’s for

AI researchers and developers focused on building autonomous agents that can interact with operating systems, manage files, and execute shell commands to solve technical problems.

Highlights

• End-to-End Pipeline: Covers everything from synthetic data generation to final evaluation.
• Large Task Corpus: Includes a published 15k task corpus in Harbor format, featuring both self-contained tasks and intricate multi-modal tasks.
• Programmatic Verification: Each task includes a verifier to objectively measure success.
• Flexible Sandboxing: Supports various execution environments, including local Docker and the Daytona cloud sandbox.