camel-ai/loong

What it solves

Project Loong addresses the challenge of scaling reasoning-capable models by enabling them to bootstrap their own intelligence. It focuses on generating high-quality synthetic data and verifying it to create a self-improvement loop for LLM agents, reducing the reliance on massive human-annotated datasets for complex reasoning tasks.

How it works

The project implements an agent-environment loop consisting of three main components:

1. Generator: Creates synthetic questions, rationales, and answers based on small, high-quality seed datasets.
2. Verifier: Evaluates the correctness of the generated responses, often by executing rationale code and comparing the output to a known answer.
3. Trainable Agent: Iteratively learns from the verified Q&A pairs using reinforcement learning (RL) and other advanced strategies to improve its reasoning capabilities.

Who it’s for

It is designed for AI researchers and developers interested in synthetic data generation, reinforcement learning, and the development of reasoning-capable LLM agents.

Highlights

• Diverse Seed Datasets: Includes over 8,700 questions across 12 domains, including advanced math, physics, chemistry, medicine, and programming.
• Automated Verification: Uses verifiers to ensure the accuracy of synthetic data through code execution.
• Modular Cookbooks: Provides reusable scripts for few-shot prompting, synthetic data generation, and exporting data for supervised fine-tuning or RL.
• Collaborative Framework: An open-source initiative encouraging contributions of new seed datasets and verifiers.