a-ghorbani/pocketpal-ai

PocketPal AI – Private, offline AI on your phone

What it is – PocketPal AI is a free, open‑source React‑Native mobile app that lets you run large language models (LLMs) and neural text‑to‑speech entirely on iOS or Android devices. No accounts, no cloud endpoints, and no internet are required after you download a model.

Key capabilities

• On‑device chat – Supports GGUF‑quantized models (Gemma, Qwen, Phi, Llama, etc.) via the llama.cpp engine (llama.rn bridge). The inference runs on the phone’s CPU, GPU (Metal / OpenCL) or Qualcomm Hexagon NPU, falling back gracefully when a backend isn’t available.
• Neural TTS – On‑device speech synthesis using ONNX Runtime (Kokoro and other voice models) – no external API calls.
• Pals & PalsHub – Create “personalities” (Assistant or Roleplay) with custom system prompts and model choices. Share or purchase community‑made Pals through the in‑app marketplace at palshub.ai.
• Tool (Talent) use – The built‑in AgentRunner lets the model call tools (calculator, date‑time, HTML renderer) during a conversation, enabling simple agentic loops.
• Model management – Browse and download GGUF models directly from Hugging Face (including gated models via a personal token) and store them locally.
• Benchmarking & leaderboard – Measure tokens‑per‑second and memory usage, optionally submit results to the public AI‑Phone leaderboard.
• Localization – UI available in 11 languages; full iPad support.

How to get started

1. Install from the App Store or Google Play (links in the README).
2. Open the app → menu ☰ → Models → pick a model that fits your device’s RAM/storage and tap Download (or add one from Hugging Face).
3. Tap Load and start chatting on the Chat screen.

Developer‑focused details

• Stack – React‑Native (v0.82.1) + TypeScript, UI via React‑Native‑Paper, state with MobX, persistence with WatermelonDB. Native bridges connect JavaScript to:
  • llama.rn (J​SI binding for llama.cpp GGUF inference)
  • react-native-speech + ONNX Runtime for TTS
• Hardware back‑ends – CPU (fallback), GPU (Metal on iOS, OpenCL on Android), NPU (Qualcomm Hexagon). The engine automatically selects the fastest available path.
• Extending the app – Add a new Talent (tool) by implementing a TalentEngine and registering it under src/services/talents/. Add a new TTS engine under src/services/tts/engines/ or a new locale JSON file under src/locales/.
• Build instructions –

  git clone https://github.com/a-ghorbani/pocketpal-ai
  cd pocketpal-ai
  nvm use               # Node 22.21.0 (pinned in .nvmrc)
  yarn install          # install JS deps
  (cd ios && pod install)   # iOS only
  yarn start            # Metro bundler
  yarn ios   # or yarn android
  

  Run yarn lint && yarn typecheck && yarn test before submitting PRs.
• Quality gates – ESLint, TypeScript type‑checking, Jest tests, locale validation, and Conventional‑Commit enforcement via Husky.

Community & support

• Discussions, bug reports, and feature requests live on GitHub Discussions and Issues.
• Translations are coordinated on Weblate.
• Sponsorship via GitHub Sponsors helps keep the app ad‑free.

License – MIT (see LICENSE).

---

Bottom line – PocketPal AI is a fully offline, privacy‑first mobile AI assistant that brings real LLM inference and neural speech synthesis to the phone you already own, and it provides a clean, extensible codebase for developers who want to experiment with on‑device agents, tool use, and custom personas.