qualcomm/GenieX

Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code

What it solves

GenieX provides a unified way to run large language models (LLMs) and vision-language models (VLMs) locally on Qualcomm Snapdragon devices. It removes the complexity of managing different hardware accelerators by providing a single runtime that handles execution across the NPU, GPU, and CPU.

How it works

GenieX acts as an inference engine that sits on top of a C SDK. It supports two primary execution paths: it can run GGUF-formatted models (from sources like Hugging Face or ModelScope) via a llama.cpp runtime, or it can run pre-compiled model bundles from the Qualcomm AI Hub for peak performance on the NPU. Developers can interact with this engine through a command-line interface, Python, Kotlin/Java (for Android), Docker, or an OpenAI-compatible local server.

Who it’s for

This project is for developers building AI applications for Qualcomm-powered hardware, including Windows ARM64 PCs, Android mobile devices, and Linux-based IoT systems. It is designed for those who need to deploy models locally while maintaining high performance across diverse Qualcomm chipsets.

Highlights

  • Supports a wide range of models, including GGUF files and pre-compiled Qualcomm AI Hub bundles.
  • Provides multiple interfaces: CLI, Python, Android SDK, Docker, and an OpenAI-compatible API server.
  • Enables hardware-accelerated inference on the Hexagon NPU, Adreno GPU, and CPU.
  • Offers a consistent developer experience across Windows, Android, and Linux platforms.
  • Includes a C SDK that serves as the foundation for all provided language bindings.

Related

  • Project
    microsoft/onnxruntime-genai

    ONNX Runtime GenAI is a Microsoft library that runs generative AI models (like LLMs) on-device using ONNX Runtime. It handles the full generation loop — preprocessing, inference, token selection, cache management, and grammar rules — and supports many model architectures, APIs (Python, C#, C/C++, Java), OSes, and hardware accelerators. It powers Foundry Local, Windows ML, and the VS Code AI Toolkit. The README includes a Python sample for Phi-3, versioning guidance, telemetry disclosure, and contribution details.

  • Project
    qualcomm/ai-hub-apps

    Qualcomm AI Hub Apps provides sample Android, Windows, and Ubuntu applications plus tutorials that demonstrate how to run vision, audio, and language models on Snapdragon devices using TensorFlow Lite, ONNX, and Qualcomm’s Genie SDK, with optional CPU/GPU/NPU acceleration.

  • Project
    mybigday/llama.rn

    A React Native binding for llama.cpp that lets mobile apps run large language models locally (chat, embeddings, reranking, multimodal vision/audio, tool calling, structured output) with GPU/NPU acceleration — no server required.

  • Project
    dineshsoudagar/local-llms-on-android

    A privacy-focused Android app that runs LLMs locally on your phone, serving as both an on-device chatbot and a local network server for AI tasks.

  • Project
    mlc-ai/web-llm

    WebLLM is a high-performance inference engine that enables large language models to run directly in web browsers using WebGPU, removing the need for server-side processing.