ztachip/ztachip

Opensource software/hardware platform to build edge AI solutions deployed on FPGA or custom ASIC hardware.

What it solves

It addresses the memory bandwidth bottleneck in edge AI inference. While most edge devices struggle to run large vision and language models (LLMs/VLMs) because the processor spends too much time waiting for data from memory, ztachip provides a data-aware architecture designed to maximize memory efficiency.

How it works

ztachip is an open-source AI accelerator for RISC-V that can be deployed on low-cost FPGAs (like the Artix-7) or custom ASICs. It utilizes a data-aware design to move model weights and data more efficiently than general-purpose processors. The system includes its own RTL, compiler, and software stack, allowing it to run classical computer vision tasks, neural network inference, and transformer models on the same hardware.

Who it’s for

It is designed for developers and hardware engineers who want to run AI inference at the edge without relying on cloud servers. It is accessible to those with no hardware knowledge via a MicroPython interface, as well as advanced users who can port the port the design to other FPGAs, ASICs, or SoCs.

Highlights

  • High Efficiency: 3.5x more efficient in memory bandwidth usage compared to a Raspberry Pi 5.
  • Versatile Workloads: Supports classical computer vision (edge detection, optical flow), TensorFlow models, and LLMs/VLMs.
  • Easy Programming: Programmable via MicroPython, removing the need for deep hardware expertise.
  • Open Source: Entire stack—including RTL, compiler, and software—is available under the Apache 2.0 license.

Related

  • Project
    alibaba/rtp-llm

    A high-performance LLM inference acceleration engine developed by Alibaba that optimizes throughput and latency for production-scale deployments.

  • Project
    google-coral/coralnpu

    An open-source RISC-V based hardware accelerator IP designed for ML inferencing in ultra-low-power wearable devices.

  • Project
    flashinfer-ai/flashinfer

    FlashInfer is a Python library that supplies ultra‑fast, hardware‑aware CUDA kernels for LLM inference—covering attention, GEMM, MoE, sampling, and related ops. It automatically selects the best backend (FlashAttention‑2/3, cuDNN, CUTLASS, TensorRT‑LLM), supports low‑precision FP8/FP4, works with all modern NVIDIA GPUs, and is already used in major serving frameworks like vLLM and TensorRT‑LLM.

  • Project
    NVIDIA/TensorRT-LLM

    An optimization library for LLMs and visual generation models that uses specialized kernels and an efficient runtime to maximize inference throughput and minimize latency on NVIDIA GPUs.

  • Project
    UbiquitousLearning/mllm

    A lightweight multimodal LLM inference engine optimized for mobile and edge devices, supporting various NPUs and GPUs with advanced quantization.