jiupinjia/rocket-recycling

What it solves

This project provides a simulation environment and reinforcement learning (RL) agents to solve the complex physics problem of rocket recovery, specifically focusing on achieving stable hovering and precision landing similar to SpaceX rocket recoveries.

How it works

The system uses a lightweight 2D rocket simulator where a rigid body with a thrust-vectoring engine is controlled by an RL agent. The agent is trained using Proximal Policy Optimization (PPO) with Generalized Advantage Estimation (GAE). It processes physical state encodings (scaled position, velocity, and orientation) to select from discrete actions combining thrust levels and nozzle angular velocities. The training process incorporates quadratic aerodynamic drag and a redesigned reward system that guides the rocket based on altitude, descent speed, and lateral alignment to prevent crashes and ensure a soft touchdown.

Who it’s for

It is designed for researchers, students, or developers interested in reinforcement learning, control theory, and aerospace simulation who want to experiment with PPO agents in a physics-based environment.

Highlights

• PPO-Clip Implementation: Uses normalized advantages, entropy regularization, and KL-based early stopping for stable training.
• Batched Training: Collects transitions from 32 independent environments to improve computational throughput.
• Physics-Based Simulation: Includes quadratic aerodynamic drag and a thrust-vectoring engine model.
• Live Monitoring: Features real-time reward curves, moving averages, and periodic simulation previews during training.