meta-recsys/generative-recommenders

What the project does

Generative Recommenders is a research‑grade code base that implements the ideas from the ICML 2024 paper Actions Speak Louder than Words: Trillion‑Parameter Sequential Transducers for Generative Recommendations. The authors show that the classic deep‑learning recommendation models (DLRMs) can be reframed as a generative modeling problem. By doing so they introduce new training and inference algorithms – notably HSTU (a high‑speed sequential transducer) and M‑FALCON – that dramatically speed up large‑scale sequential recommendation (10×‑1000× faster) while improving accuracy.

The repository contains:

• Model definitions and training scripts for the HSTU family and a DLRM‑v3 built on top of it.
• Configuration files (Gin) for reproducing experiments on public datasets (MovieLens‑1M, MovieLens‑20M, Amazon Books).
• Utilities for generating a synthetic 3‑billion‑rating dataset via fractal expansion.
• Highly‑optimized CUDA/Triton kernels (FlashAttention‑V3‑based) for ultra‑fast attention and FP8 matrix‑multiply, plus benchmarks that report speed‑ups on NVIDIA H100 GPUs.
• End‑to‑end scripts for training and inference on multi‑GPU setups.

In short, it is a full research implementation that lets you train, evaluate, and benchmark state‑of‑the‑art generative recommender models at scale.

---

How to get started

1. Install dependencies – the easiest way is pip3 install -r requirements.txt. The README notes it has been tested on Ubuntu 22.04 with CUDA 12.4 and Python 3.10. If you prefer a manual install, you need PyTorch plus gin-config, pandas, fbgemm_gpu, torchrec, and tensorboard.
2. Prepare data – run python3 preprocess_public_data.py which creates a tmp/ folder and downloads the MovieLens / Amazon data.
3. Run an experiment – pick a Gin config (e.g., configs/ml-1m/hstu-sampled-softmax-n128-large-final.gin) and launch:

   CUDA_VISIBLE_DEVICES=0 python3 main.py \
       --gin_config_file=configs/ml-1m/hstu-sampled-softmax-n128-large-final.gin \
       --master_port=12345
   

   Logs are written to exps/; you can visualise them with TensorBoard.
4. Benchmark kernels – for low‑level performance testing, run the provided benchmark modules, e.g.:

   python3 -m generative_recommenders.ops.benchmarks.hstu_ultra_stack_bench \
       --config-name hstu_ultra_semi_local --provider all --sequence-lengths 512
   

5. Scale up – the repo includes scripts to generate a synthetic 3 B‑rating dataset (run_fractal_expansion.py) and to train/infer with 4 GPUs for the DLRM‑v3 model.

---

Who might find this useful

Audience	Why it matters
Academic researchers in recommender systems or large‑scale generative models	Full reproducible pipeline, state‑of‑the‑art results, and novel algorithmic contributions (HSTU, M‑FALCON).
Engineers building production recommender services	Optimised CUDA/Triton kernels, FP8 support, and multi‑GPU training scripts that can be adapted for real‑world workloads.
Performance engineers / hardware architects	Detailed benchmarks on H100 GPUs, FlashAttention‑V3‑based kernels, and FP8 quantisation pathways for measuring speed‑up potential.
Students learning about sequential recommendation	End‑to‑end example code (data preprocessing, model config, training loop) that bridges theory (the paper) and practice.

---

Limitations / open challenges

• Hardware‑heavy – the fastest kernels require high‑end GPUs (e.g., H100 with ≥24 GB HBM). Running the full‑scale experiments on modest hardware will be slow or infeasible.
• Synthetic data reliance – the 3 B‑rating dataset is generated via fractal expansion, which may not reflect real‑world distributional nuances.
• Research‑grade code – while the repository is functional, it lacks the polish of a production library (e.g., extensive testing, CI, or easy pip installable package).
• Scope limited to sequential recommendation – the generative formulation is demonstrated for next‑item prediction; extending to other recommendation tasks (e.g., multi‑modal, graph‑based) would require additional work.

---

How it fits into the broader AI ecosystem

Generative Recommenders sit at the intersection of large‑scale language‑style generative modeling and industrial recommender systems. By treating recommendation as a sequence‑to‑sequence generation problem, the project leverages advances from the transformer/LLM world (e.g., sampled softmax, FP8 quantisation, FlashAttention) to push the limits of recommendation throughput and model size. This mirrors a broader trend where techniques originally built for NLP (large‑parameter generative models, efficient attention kernels) are being repurposed for other domains such as recommendation, search, and even robotics. The work also contributes to the ongoing discussion on compute scaling laws for recommender systems, showing that trillion‑parameter models can be trained and served efficiently with the right algorithmic and systems tricks.

---

Quick reference (commands you’ll use most)

# Install
pip3 install -r requirements.txt

# Preprocess public data
mkdir -p tmp/ && python3 preprocess_public_data.py

# Train HSTU on MovieLens‑1M (single GPU)
CUDA_VISIBLE_DEVICES=0 \
python3 main.py \
  --gin_config_file=configs/ml-1m/hstu-sampled-softmax-n128-large-final.gin \
  --master_port=12345

# Visualise results
tensorboard --logdir ~/generative-recommenders/exps/ml-1m-l200/ --port 24001

# Benchmark ultra‑fast attention (H100)
python3 -m generative_recommenders.ops.benchmarks.hstu_ultra_stack_bench \
  --config-name hstu_ultra_semi_local --provider all --sequence-lengths 512

---

License

Apache 2.0 – free for commercial and academic use.