deepseek-ai/DeepSelect

DeepSelect: TopK kernels for DeepSeek Sparse Attention (DSA) and Samplers

What it solves

DeepSelect provides a high-performance implementation of the TopK kernel, which is used for DeepSeek Sparse Attention (DSA) and sampling. It aims to replace the standard torch.topk function, which can be a bottleneck in large-scale model inference and training, by offering significantly faster execution speeds.

How it works

DeepSelect implements optimized kernels for NVIDIA CUDA and Huawei Ascend platforms. It focuses on two primary scenarios: a "Lightning Indexer" scenario for bfloat16 inputs with small TopK values (up to 4096), and a "Sampling" scenario for float32 inputs with large vocabularies (around 128K) and small TopK values. The implementation optimizes memory bandwidth usage, as TopK operations are essentially memory-bound rather than compute-bound.

Who it’s for

This tool is designed for developers and researchers working with DeepSeek models (specifically V3.2, V4, and V4.1) or any AI project requiring high-performance TopK operations on CUDA or Ascend NPUs.

Highlights

  • Significant Speedup: Achieves 2x to 20x faster performance compared to vanilla torch.topk.
  • Multi-Platform Support: Compatible with both NVIDIA CUDA and Huawei Ascend platforms.
  • Hardware-Specific Optimization: Tailored for specific data types (bfloat16 and float32) and common LLM workloads like sparse attention and sampling.
  • Flexible Output: Allows users to disable sorting by index or skip value output to further increase performance.

Related

  • Project
    deepseek-ai/FlashMLA

    FlashMLA is a high-performance library of optimized attention kernels for DeepSeek models, providing accelerated sparse attention, quantized KV cache support, and fused operations for NVIDIA and Huawei hardware.

  • Project
    MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark

    A deployment recipe for running DeepSeek-V4-Flash-Vision-Exp on two-node DGX Spark clusters, featuring 1M-token context support, speculative decoding, and native image processing.

  • Project
    MoonshotAI/FlashKDA

    A high-performance CUDA kernel implementation of Kimi Delta Attention (KDA) built on CUTLASS for NVIDIA SM90+ GPUs, serving as an optimized backend for the flash-linear-attention library.

  • Project
    Dao-AILab/quack

    A collection of high-performance GPU kernels for deep learning operations like RMSNorm and Softmax, implemented using CuTe-DSL for modern NVIDIA GPUs.

  • Project
    lightseekorg/tokenspeed

    A high-performance LLM inference engine optimized for agentic workloads that separates its C++ control plane from its Python execution plane for maximum speed and flexibility.