a-ghorbani/pocketpal-ai

An app that brings language models directly to your phone.

PocketPal AI – an on‑device AI chat assistant

What it is – PocketPal AI is a free, open‑source mobile app (iOS & Android) that lets you talk to large language models, hear them speak, and give them simple tools (calculator, date‑time, HTML rendering) – all without ever leaving your phone. The app downloads a GGUF‑format model from Hugging Face once, then runs inference locally using the phone’s CPU, GPU (Metal/OpenCL) or NPU (Qualcomm Hexagon). No cloud account, no subscription, and everything stays private unless you explicitly share benchmark results or feedback.

Core features

Feature What you get
On‑device chat Run popular LLMs (Gemma, Qwen, Phi, Llama, etc.) offline in GGUF format.
Neural TTS On‑device text‑to‑speech (Kokoro and other ONNX models) – no internet calls.
Pals & PalsHub Create or download personalized “assistants” with custom prompts, personalities, and even role‑play settings.
Tool use (Talents) Models can call built‑in tools (calculator, date‑time, HTML renderer) via a tool‑use loop.
Hardware acceleration Inference can use CPU, GPU (Metal on iOS, OpenCL on Android) or NPU; the app falls back gracefully.
Benchmark & leaderboard Measure tokens‑per‑second and memory usage; optionally submit results to the public AI‑Phone leaderboard.
Localization UI available in 11 languages; full iPad support.
Open‑source & free MIT‑licensed, no hidden “pro” tier.

How it works (high‑level)

  1. UI layer – React Native app (React Native Paper, MobX state, WatermelonDB for chat history). The AgentRunner streams model tokens and dispatches Talents when the model requests them.
  2. Bridging layer – Native modules expose llama.cpp inference (llama.rn) and ONNX‑Runtime TTS to JavaScript.
  3. Engine layer – llama.cpp runs quantized GGUF LLMs; ONNX Runtime runs neural‑TTS models.
  4. Hardware layer – Math is executed on the device’s CPU, GPU (Metal/OpenCL), or NPU (Qualcomm Hexagon) with automatic fallback.

Getting started as a user

  1. Install from the App Store or Google Play.
  2. Open the app → Menu ☰ → Models → pick a model that fits your device’s memory and download it (or add one from Hugging Face with a personal token).
  3. Load the model and start chatting. You can enable voice output, switch between Pals, or run the benchmark screen.

Getting started as a developer

  • Prerequisites – Node 22 (via .nvmrc), Yarn 1, Xcode + CocoaPods (iOS) or Android Studio (Android). The repo follows a standard React‑Native layout.
  • Build – git clone … && nvm use && yarn install && (cd ios && pod install) then yarn ios or yarn android.
  • Code structure –
    • src/screens – UI pages (Chat, Models, Pals, Benchmark, Settings).
    • src/services/agent – AgentRunner implements the chat‑tool loop.
    • src/services/talents – built‑in tools; add new ones by implementing a TalentEngine and registering it.
    • src/services/tts – TTS engines; extendable with additional ONNX models.
    • src/store – MobX stores for models, chat sessions, settings, etc.
  • Quality gates – yarn lint, yarn typecheck, yarn test must pass before PRs.
  • Extending – Add new Talents, TTS engines, or locales (JSON files under src/locales).

Who might benefit?

  • Privacy‑conscious users who want a conversational AI that never uploads data.
  • Travelers / field workers needing offline AI on a plane or remote location.
  • Developers interested in on‑device LLM inference, hardware acceleration, or building custom AI‑powered assistants.

License & community

  • License – MIT.
  • Support – GitHub Discussions, Issues, the website pocketpal.dev, and optional sponsorship via GitHub Sponsors.
  • Acknowledgements – Built on llama.cpp, llama.rn, ONNX Runtime, React‑Native, MobX, WatermelonDB, and many other open‑source libraries.

Bottom line – PocketPal AI demonstrates that modern quantized LLMs can run fully offline on a smartphone, offering a private, extensible AI assistant without any recurring fees.

Related

  • Project
    pocketpaw/pocketpaw

    A self-hosted AI agent that runs locally on your machine, providing a native desktop app and integration with multiple messaging platforms like Discord and Slack.

  • Project
    Chuloo/mural

    Mural is a native iPhone/Android app that uses your own OpenAI API key to run AI‑driven spoken conversations for language learning. It stores progress locally, adapts difficulty from your replies, and supports 12 languages out of the box.

  • Project
    AtomicBot-ai/Atomic-Chat

    Atomic Chat is a desktop/mobile app that runs open‑weight LLMs locally and exposes them via an OpenAI‑compatible HTTP server (`http://localhost:1337/v1`). It supports multiple back‑ends (llama.cpp, a TurboQuant‑enhanced fork, and Apple‑Silicon MLX‑VLM), speculative decoding tricks for speed, cloud‑model fallbacks, and one‑click integration with many open‑source coding agents. The UI handles model download, engine selection, and API routing, making private on‑device inference easy for developers and researchers.

  • Project
    dineshsoudagar/local-llms-on-android

    An Android application that enables fully offline, private LLM inference on-device and can act as a local LAN server for other devices via an OpenAI-compatible API.

  • Project
    kyutai-labs/pocket-tts

    Pocket TTS is a 100 M‑parameter, CPU‑only text‑to‑speech library that streams audio with ~200 ms latency, supports multiple languages and voice cloning, and offers a Python API, CLI, and local HTTP server. It can be installed via pip (or uv) and runs faster than real‑time on modest hardware, with community‑trained models and several alternative runtimes for browsers and embedded devices.