Ch921-cell/Remember-R1

What it solves

Remember-R1 addresses the problem of "long-context visual forgetting," where multimodal models struggle to maintain and utilize visual information throughout extended reasoning processes.

How it works

The project uses reinforcement learning to train models by augmenting standard answer correctness rewards with three specific process-level rewards:

• Visual Vocabulary Reward: Encourages the model to explicitly express visual evidence in its reasoning.
• Visual Memory Reward: Encourages the model to keep relying on visual data in later steps of the reasoning trajectory.
• Visual Key-Region Reward: Encourages the model to focus its attention on regions of the image that are relevant to the question.

Who it’s for

This is primarily for AI researchers and developers working on multimodal large language models (MLLMs) and visual reasoning tasks that require long-context memory.

Highlights

• Implements a reinforcement learning framework to mitigate visual forgetting.
• Provides pre-trained model checkpoints in 3B and 7B parameter sizes.
• Includes a dedicated dataset for training and evaluation.
• Integrated with VLMEvalKit for standardized performance assessment.