AkaliKong/MiniOneRec

Minimal reproduction of OneRec

What it solves

MiniOneRec addresses the challenge of scaling generative recommendation systems. It provides a full end-to-end pipeline to transform traditional recommendation tasks into a generative process where a Large Language Model (LLM) predicts the next item as a discrete token, rather than relying on traditional ranking methods.

How it works

The framework operates in three primary stages:

  1. SID Construction: Every product is converted into a "Semantic ID" (SID)—a compact, semantically meaningful token. This is done by encoding item titles and descriptions through a frozen text encoder and quantizing the embeddings using methods like RQ-VAE or RQ-Kmeans.
  2. Supervised Fine-Tuning (SFT): The LLM is trained to treat user history as a sequence of SIDs and predict the next SID. This stage includes language-alignment objectives to ensure the model can map between natural language and the SID space.
  3. Recommendation-Oriented RL: The model is further refined using Group Relative Policy Optimization (GRPO). It uses constrained beam search to ensure generated tokens are valid SIDs and a reward signal that combines binary correctness with rank-aware penalties to improve precision and diversity.

Who it’s for

It is designed for AI researchers and developers building generative recommendation systems who want a standardized workflow for SID creation, SFT, and reinforcement learning.

Highlights

  • End-to-End Pipeline: Covers everything from raw data preprocessing and SID generation to final RL polishing.
  • Flexible SID Generation: Supports multiple quantization methods including RQ-VAE, RQ-Kmeans, and constrained RQ-Kmeans.
  • Constrained Decoding: Implements a logit processor to guarantee that the model only generates valid item IDs.
  • RL Integration: Uses GRPO to optimize recommendations based on correctness and ranking metrics.