MINT-SJTU/Evo-RL

Evo‑RL – Real‑world Reinforcement Learning Toolkit

What it is – Evo‑RL is an open‑source framework for doing offline and online reinforcement learning on real‑world robot platforms. It builds on the LeRobot library (a Hugging Face project for robot data collection and policy training) and adds a full pipeline for:

1. Hardware integration – supports Seeed Studio SO‑101 arms and AgileX PiPER / PiPER‑X robots, with detailed instructions for serial, CAN‑bus, and camera wiring.
2. Data collection – command‑line tools (lerobot‑human‑inloop‑record) that let a human tele‑operate the robot while automatically logging multi‑modal observations (RGB, RealSense depth, joint states) to a Hugging Face dataset.
3. Value‑function training & inference – train a return‑to‑go estimator (default pistar06) on the collected dataset, write back value, advantage, and binary “good‑trajectory” tags.
4. Policy training – train a language‑conditioned policy that consumes the advantage‑conditioned tags as part of the task description, using the standard lerobot‑train script.
5. Closed‑loop rollout – deploy the trained policy in a human‑in‑the‑loop loop to collect the next round of data, enabling iterative improvement.

The repository ships with scripts, configuration templates, and a Docker‑free conda install, plus a public RW‑RL dataset on Hugging Face.

---

Quick‑Start Summary

# Clone & set up the environment
git clone https://github.com/MINT-SJTU/Evo-RL.git && cd Evo-RL
conda create -y -n evo-rl python=3.10
conda activate evo-rl
pip install -e .   # installs the package and CLI entry points

From there you can:

• Verify hardware with lerobot-teleoperate (see the README for the exact command strings for single‑arm or bimanual setups).
• Record a dataset with lerobot-human-inloop-record.
• Train a value model with lerobot-value-train.
• Infer value tags with lerobot-value-infer.
• Train a policy with lerobot-train.
• Run a closed‑loop rollout using the same lerobot-human-inloop-record command but adding --policy.path=<checkpoint>.

---

Key Features Grounded in the README

Area	What the repo provides
Hardware support	Detailed configuration for SO‑101 (serial) and AgileX PiPER/PiPER‑X (USB‑CAN) robots, including stable device‑path recommendations and calibration file layout.
Dataset handling	One‑click push to Hugging Face (--dataset.push_to_hub=true), automatic dataset reports (lerobot-dataset-report), and a public RW‑RL dataset.
Value function	Training script (lerobot-value-train) with bfloat16 support, multi‑GPU launch via accelerate, and a plug‑in architecture for custom value models.
Advantage‑Conditioned Policy (ACP)	After value inference, three new columns (value_<TAG>, advantage_<TAG>, acp_indicator_<TAG>) are added to the dataset; the policy training script can ingest the binary indicator as a text tag, with dropout control.
End‑to‑end pipeline	The README walks through all seven stages—from installation to the next data‑collection round—so users can reproduce the full research loop.
Community & reproducibility	Links to a project website, WeChat group, upcoming paper, and Apache‑2.0 license; all code, models, and datasets are meant to be openly reusable.

---

Who Might Use This

• Robotics researchers who need a ready‑made stack for real‑world RL on low‑cost manipulators.
• ML engineers looking to experiment with advantage‑conditioned training on robot data.
• Educators who want a hands‑on lab that goes from tele‑operation data collection to policy deployment on actual hardware.

---

Limitations Mentioned

• The paper and pre‑trained models are listed as “Coming Soon,” so the repository currently ships mainly with the pipeline code and the RW‑RL dataset.
• Hardware‑specific steps (e.g., calibrations, CAN‑bus serial IDs) require access to the supported robots; the repo does not provide simulation fallbacks.

---

Citation & License

• License: Apache‑2.0 (per LICENSE badge).
• Citation: A paper is promised in the future; users can cite the repository URL for now.

---

All information above is taken directly from the repository’s README; no external assumptions have been added.