dineshsoudagar/local-llms-on-android

What it solves

It provides a way to run Large Language Models (LLMs) entirely on Android devices, ensuring user privacy and offline availability. It eliminates the need for cloud-based AI by allowing users to download, switch, and manage models directly on their phones.

How it works

The app uses a small base APK and allows users to download specific model weights (such as Qwen and Gemma) after installation. It leverages the ONNX and LiteRT backends to enable hardware acceleration via GPU and NPU on supported devices, reducing latency. It supports multiple input modalities, including text, voice, and images (via OCR or native image support).

Who it’s for

Android users who want a private, offline AI assistant that can run on a variety of hardware, from low-end to flagship mobile devices.

Highlights

• Fully On-Device: All inference, OCR, and image processing happen offline after the initial model download.
• Flexible Model Management: Users can download, switch between, and delete multiple models (e.g., Gemma 4, Qwen 3) to manage device storage.
• Multimodal Input: Supports voice prompting, camera capture with cropping, and image input via OCR or native Gemma image support.
• Hardware Optimized: Uses LiteRT for GPU and NPU acceleration to improve speed and reduce latency.
• Customizable: Includes editable model instructions, presets, and UI themes (dark/light mode).