dmlc/dlpack

common in-memory tensor structure

What it solves

DLPack provides a standardized, open in-memory tensor structure. This solves the problem of tensors being locked into specific deep learning frameworks, which makes it difficult to share operators or swap backend implementations across different libraries.

How it works

Instead of implementing its own tensors or operators, DLPack acts as a common bridge. It defines a standardized way to represent tensors in memory so that different frameworks can exchange data without needing complex conversions or redundant copies.

Who it’s for

It is designed for developers of deep learning frameworks and vendor-level operator implementations who need to share tensors and operators across different software ecosystems.

Highlights

  • Standardized in-memory tensor structure for cross-framework sharing.
  • Simplifies the wrapping of vendor-level operator implementations.
  • Allows for quick swapping of backend implementations, such as different BLAS versions.
  • Enables final users to mix and match operators from different frameworks.

Related

  • Project
    hpcaitech/ColossalAI

    A distributed deep learning framework that makes training and inference for large AI models faster and cheaper through advanced parallelism and memory management.

  • Project
    intelligent-machine-learning/dlrover

    An automatic distributed deep learning system that simplifies large-scale AI model training by providing automated fault tolerance, auto-scaling, and rapid checkpoint recovery on K8s and Ray.

  • Project
    Mesh-LLM/mesh-llm

    A distributed LLM serving system that pools GPUs and memory across multiple machines to run large models via an OpenAI-compatible API.

  • Project
    mosaicml/composer

    A PyTorch-based deep learning library that simplifies distributed training at scale, providing tools for memory optimization and algorithmic speedups to reduce training time and cost.

  • Project
    apache/tvm-ffi

    An open ABI and FFI for machine learning systems that enables kernel libraries and DSLs to work across multiple frameworks and languages with zero-copy interop.