zhengkid/Dream-RSI

What it solves

Recursive self-improvement (RSI) in AI agents often struggles with inefficient exploration. As targets become more complex, agents waste significant computation on ineffective search directions because fixed exploration strategies cannot adapt, and optimizing these strategies online is too expensive due to delayed and costly feedback.

How it works

Dream-RSI treats accumulated discovery history as a replay simulator. Instead of rerunning expensive discovery agents or evaluators, the system uses a recorded tree of past exploration decisions and their outcomes to create a "world" the agent can "dream" in.

The process follows a continuous loop:

1. Online Exploration: A current policy drives discovery and logs all traces.
2. Simulator Construction: These recorded trees are converted into a reusable simulator pool.
3. Dreaming-based Improvement: Candidate policies are evaluated and refined by simulating paths through the existing pool, providing immediate, low-cost off-policy feedback.
4. Redeployment: The improved policy is deployed back online to further expand the pool.

Who it’s for

Researchers and developers working on recursive self-improvement, automated program discovery, and meta-learning for AI agents tasked with complex optimization or engineering problems.

Highlights

• History as Replay Simulator: Converts past discovery logs into a simulator, making delayed feedback reusable for efficient policy evaluation.
• Meta-layer RSI Loop: Implements a self-improvement cycle specifically at the exploration layer without needing to change the underlying coding agent.
• Proven Efficiency: Demonstrated improvements in algorithm engineering, mathematical optimization, and GPU kernel engineering, reducing discovery compute and increasing performance.