off-grid-ai/OGAM

What it solves

Off Grid AI provides a complete, private AI suite that runs entirely on-device for Android, iOS, and macOS. It eliminates the need for cloud-based AI services, ensuring that zero data leaves the device, which solves the problem of data privacy and connectivity requirements for AI tools.

How it works

The application leverages native hardware acceleration (Adreno GPUs via OpenCL, Apple Silicon Metal, and Snapdragon Hexagon NPUs) to run a variety of local models. It uses llama.cpp and whisper.cpp for text and voice, and Core ML or NPU-accelerated Stable Diffusion for images. It includes a model manager to handle finite device RAM and a local SQLite-based RAG system for document analysis using a bundled MiniLM model for embeddings.

Who it’s for

Users who prioritize data privacy, those who need AI capabilities without an internet connection, and developers or enthusiasts who want to run various GGUF models and vision-language models on their mobile devices or Macs.

Highlights

• Multimodal Capabilities: Supports text generation (Llama, Qwen, Phi), image generation (Stable Diffusion), vision AI (SmolVLM, Qwen-VL), and voice transcription (Whisper).
• Local RAG: On-device document analysis via a project knowledge base that chunks and embeds PDFs and text files locally.
• Hardware Acceleration: Automatic detection and use of GPU/NPU for faster inference on flagship devices.
• Tool Calling: Built-in tools for web search, calculator, and knowledge base search with runaway prevention.
• Remote Server Support: Ability to connect to OpenAI-compatible local servers like Ollama or LM Studio.
• Memory Management: Granular control over RAM usage with policies like Lean, Balanced, and Aggressive loading.