qualcomm/GenieX

Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code

What it solves

GenieX simplifies the process of running frontier Large Language Models (LLMs) and Vision Language Models (VLMs) locally on Qualcomm Snapdragon devices. It removes the complexity of targeting specific hardware by providing a unified interface to leverage the Hexagon NPU, Adreno GPU, or CPU for on-device inference.

How it works

GenieX acts as an inference runtime that wraps two different backends: llama.cpp for broad compatibility with GGUF models from Hugging Face, and the Qualcomm AI Engine Direct (qairt) for optimized, pre-compiled bundles from the Qualcomm AI Hub. It provides a single C SDK that is exposed through multiple high-level interfaces, including a CLI, Python library, Kotlin/Java for Android, Docker, and an OpenAI-compatible server.

Who it’s for

Developers building AI applications for Windows ARM64 (Snapdragon X/X Elite), Android (Snapdragon 8 Elite), and Linux ARM64 (IoT) devices who want to run models locally without writing low-level hardware code.

Highlights

  • Multi-Backend Support: Switch between GGUF models for flexibility and pre-compiled bundles for peak NPU performance.
  • Broad Interface Options: Accessible via Python (mirroring Hugging Face transformers), CLI, Android SDK, or a drop-in OpenAI-compatible API server.
  • Cross-Platform: Supports Windows ARM64, Android, and Linux ARM64.
  • Hardware Acceleration: Direct access to the Hexagon NPU, Adreno GPU, and CPU.