osolmaz/switch

What it solves

switch solves the problem of integrating "decision models"—AI models that output probabilities for specific choices rather than generating free-form text—into complex, multi-step tasks. It provides a structured way to manage these models so they can navigate tasks like document analysis or game playing without needing to generate code or tool calls themselves.

How it works

Developers define a "domain," which acts as a state machine for a specific task. The framework manages an agent loop where the model is presented with typed questions (like choosing a move or selecting a document page). The model provides a choice, and the framework's transition function updates the state accordingly. This loop repeats until the task is complete, with every step recorded in a trace file that allows for debugging or replaying the process without re-calling the model.

Who it’s for

This framework is for developers building specialized AI agents or evaluation judges who want to use efficient, narrow-domain decision models. It is ideal for those who need to perform complex tasks—such as navigating large document sets for grading or playing games—where a general-purpose chat model might be too expensive or lack the necessary precision.

Highlights

• Provides a unified interface for decision models to perform multi-step reasoning.
• Enables efficient context management by allowing models to "navigate" documents and only read relevant parts.
• Includes built-in support for replaying task runs from trace files to debug or fork logic.
• Offers a command-line harness to manage runs, view progress in a browser, and handle case folders.
• Designed for cost-efficiency by enabling the use of small, domain-specific models.