alibaba/RecIS

What it solves

RecIS is designed to handle the challenges of ultra-large-scale recommendation systems, specifically the need to unify sparse and dense computing. It addresses the performance bottlenecks associated with memory access in sparse operators and the complexities of training industrial-grade recommendation models combined with multimodal or large-scale models.

How it works

Built on the PyTorch ecosystem, RecIS uses a modular architecture to manage the training pipeline:

• ColumnIO: Handles distributed sharded data reading and feature pre-computation.
• Feature Engine: Manages feature engineering (like hashing and bucketizing) with operator fusion to reduce overhead.
• Embedding Engine: Manages scalable, conflict-free KV storage embedding tables with multi-table fusion for better memory access.
• Saver: Uses the SafeTensors standard for saving and loading sparse parameters.
• Pipelines: Orchestrates the entire training workflow, including multi-stage and multi-objective computation.

To optimize performance, it implements a two-level storage architecture (IDMap and EmbeddingBlocks) for dynamic embeddings and uses All-to-All collective communication for distributed parameter sharding and request merging.

Who it’s for

It is intended for engineers and researchers working on industrial-scale recommendation, advertising, and search systems that require high-performance training of models combining sparse features and dense deep learning components.

Highlights

• Unified Framework: Integrates sparse-dense computing within the PyTorch ecosystem.
• High Performance: Optimizes GPU utilization through operator fusion and vectorized memory access.
• Dynamic Embeddings: Implements a flexible two-level storage system for HashTable embeddings that can be placed on GPU or CPU.
• Distributed Scaling: Uses parameter aggregation and sharding to distribute large embedding tables across compute nodes.