NVIDIA/nvshmem

NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process communication and coordination overheads by allowing programmers to perform one-sided communication from within CUDA kernels and on CUDA streams.

What it solves

NVSHMEM is designed to enable scalable communication between NVIDIA GPUs in a cluster. It solves the problem of complex data movement between GPUs, allowing them to access memory across the cluster as if it were a single, shared global address space.

How it works

Based on the OpenSHMEM standard, it provides a partitioned global address space (PGAS) partitioned across NVIDIA GPUs. It allows for one-sided transfers, atomics, signaling, and synchronization. Communication can be initiated by the CPU (host), CUDA kernels (device), or CUDA streams, giving developers flexibility in how they initiate data movement.

Who it’s for

Developers building high-performance computing (HPC) applications or large-scale AI training and inference workloads that require efficient, GPU-initiated communication across multiple GPUs and nodes.

Highlights

  • own-sided communication transfers and atomics
  • Support for host, CUDA kernel, and CUDA stream interfaces
  • Integration with process launchers like Open MPI, Hydra, and Slurm
  • Python support via NVSHMEM4Py

Related

  • Project
    Project-HAMi/HAMi

    A Kubernetes-native virtualization middleware that enables sharing and isolating GPU and other AI accelerator resources across multiple workloads to increase hardware utilization.

  • Project
    Mesh-LLM/mesh-llm

    A distributed LLM serving system that pools GPUs and memory across multiple machines to run large models via an OpenAI-compatible API.

  • Project
    NVIDIA/DCGM

    A suite of tools for managing and monitoring NVIDIA data-center GPUs in cluster environments, providing health monitoring, diagnostics, and governance policies.

  • Project
    ROCm/legacy-rocm-build

    ROCm / legacy‑rocm‑build is the (now‑deprecated) repository that assembles AMD’s open‑source GPU compute stack—drivers, HIP runtime, math libraries, profiling tools, and more—so developers can run AI/ML workloads on AMD GPUs. It points users to install guides and to the newer TheRock monorepo.

  • Project
    microsoft/mscclpp

    A GPU-driven communication stack that provides customizable, 0-copy inter-GPU communication primitives to improve performance for scalable AI applications and LLM serving.