tile-ai/tilelang-ascend

Ascend TileLang adapter

What it solves

TileLang-Ascend addresses the difficulty of creating high-performance AI compute kernels for Huawei Ascend NPUs. It allows developers to write optimized kernels (such as GEMM, attention mechanisms, and vector operations) using a Pythonic syntax, removing the need to manually handle complex low-level NPU optimizations while maintaining state-of-the-art performance.

How it works

It is a domain-specific language (DSL) built on the TVM compiler infrastructure. It maps high-level Pythonic code to the Ascend NPU architecture by managing memory hierarchies (Global, L1, and L0 buffers) and providing specialized compiler backends (Ascend C & PTO and AscendNPU IR). The system supports both a "Developer mode" for automatic synchronization and scope separation, and an "Expert mode" for manual control over execution scopes and synchronization flags.

Who it’s for

AI engineers and kernel developers who need to optimize compute workloads specifically for Huawei Ascend A2 and A3 processors.

Highlights

  • Broad Operator Support: Includes implementations for GEMM, Flash Attention, Sparse Flash Attention, Convolution, and Softmax.
  • Memory Optimizations: Features automatic buffer reuse to reduce on-chip memory footprint and automatic workspace allocation.
  • Performance Tooling: Supports software pipelining (T.Pipelined), automatic vectorization (T.Parallel), and L2 cache swizzling.
  • PyTorch Integration: Provides torch_tl_ascend for seamless workflows between PyTorch and the NPU.

Related

  • Project
    deepseek-ai/DeepGEMM-Ascend

    DeepGEMM‑Ascend is an open‑source, Ascend‑NPU‑specific port of the DeepGEMM matrix‑multiply library. It offers API‑compatible BF16/FP8/FP4 GEMM, MQA, and MoE kernels, hides Ascend’s low‑level layout complexities, and claims 98‑99 % hardware utilisation on Ascend 950. Installable via pip after building a C++ extension, it’s aimed at AI engineers who need fast linear‑algebra primitives on Huawei’s hardware.

  • Project
    tile-ai/tilelang

    A concise domain-specific language and compiler built on TVM for developing high-performance GPU/CPU/NPU kernels like GEMM and FlashAttention.

  • Project
    deepseek-ai/TileKernels

    A library of highly optimized kernels for LLM training and inference, providing high-performance implementations of MoE routing, quantization, and Engram gating.

  • Project
    ztxz16/fastllm

    FastLLM is a C++‑based, PyTorch‑free inference engine for large language models. It supports dense and MoE models, hybrid GPU/CPU/NUMA/disk deployment, multi‑GPU tensor parallelism, speculative decoding (MTP/DSpark/DFlash2), many quantization formats, and offers a Python CLI, web UI, and OpenAI‑compatible API server. Install via `pip install ftllm`, run a model with `ftllm run`, or launch a service with `ftllm server`. Extensive docs cover model‑specific guides, performance tuning, and community support.

  • Project
    xLLM-AI/xllm

    An efficient LLM inference framework optimized for Chinese AI accelerators to enable high-throughput, low-latency enterprise deployment.