NVIDIA/recsys-examples

What it solves

This project provides a collection of optimized recommender system models and components designed to handle large-scale ranking, retrieval, and generative recommendation tasks while maximizing GPU performance.

How it works

It combines high-performance model architectures with specialized GPU libraries and automated optimization tools:

• Model Examples: Implements HSTU (Hierarchical Sequential Transformer Unit) for ranking and retrieval, and SID-GR (Semantic ID Generative Recommender) for generative retrieval using hierarchical semantic-ID prediction.
• GPU Libraries: Includes DynamicEmb for model-parallel dynamic embedding tables with flexible eviction policies, a RecSys KVCache Manager for user-ID-based KV-cache reuse, and specialized beam-search decode attention kernels.
• Automated Optimization: Features Talos, an agent-driven toolkit that uses coding agents (like Claude Code or Codex) to profile PyTorch models, rewrite hotspots, and verify speedups without human intervention.
• Deployment: Integrates with Triton Inference Server and AOTInductor for native C++ deployment and optimized serving.

Who it’s for

Machine learning engineers and researchers building large-scale recommendation systems who need to optimize training and inference latency on NVIDIA GPUs.

Highlights

• Agentic Optimization: Automated end-to-end PyTorch model optimization via Talos.
• Generative Recommendation: Specialized support for SID-GR training and high-beam inference.
• Advanced Embedding Management: Dynamic embedding tables with GPU/host hash-table storage and incremental synchronization.
• High-Efficiency Inference: Paged KV-cache, CUDA graph optimization, and asynchronous host KV onload/offload.