adithya-s-k/FineEnvs

What it solves

FineEnvs provides a collection of open-source, reproducible reinforcement learning (RL) environments specifically designed for training and evaluating LLM agents. It addresses the bottleneck in RL for LLMs, where designing and deploying scalable, verifiable environments is often more difficult than the RL algorithms themselves.

How it works

The project organizes its content into self-contained, end-to-end projects that combine environment implementations, training scripts, and results. It supports multiple RL frameworks (including OpenEnv, ORS, NeMo Gym, and Verifiers) and provides a variety of of environments, such as:

• LaTeX OCR: Training a VLM to read math images into LaTeX with a verifiable reward.
• Watercolour: Training a model to paint using p5.brush sketches, rewarded by an aesthetic preference model.
• Hugging Face Spaces: Many environments are deployed as live Spaces for immediate interaction.
• SmolDataEnvs: A dataset of over 5,000 verified data-analysis tasks based on Kaggle notebooks.

Additionally, it includes agent skills (via the SKILL.md spec) that allow users to generate new RL environments from plain-English descriptions across different frameworks.

Who it’s for

Researchers and developers building LLM agents who need ready-made environments for RL training, reproducible training recipes, or a guide on how to build and scalable RL environments from scratch.

Highlights

• Multi-Framework Support: Implements the same environments across six different RL frameworks to show the differences in implementation.
• End-to-End Recipes: Provides the full loop from environment design to trained model artifacts on the Hugging Face Hub.
• Verifiable Rewards: Focuses on environments with objective, non-LLM-based rewards (e.g., exact match, numeric tolerances) to avoid reward hacking.
• Automated Environment Creation: Includes tools to port environment descriptions into runnable code across four frameworks.