nobodywho-ooo/nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

What it solves

NobodyWho provides a way to run Large Language Models (LLMs) locally and offline on a wide variety of devices, including mobile phones (iOS, Android), desktop computers (Windows, macOS, Linux), and specialized hardware like Vision Pro and Apple Watch. It eliminates the need for API keys, hidden fees, and internet connectivity for AI inference.

How it works

Built with a Rust core, the engine uses llama.cpp for text, vision, and embeddings, and the ONNX Runtime for speech-to-text, text-to-speech, and Voice Activity Detection (VAD). It leverages GPU acceleration via Vulkan or Metal to ensure fast performance across different operating systems. The project provides bindings for multiple languages and frameworks, including Python, Kotlin, Swift, React Native, Flutter, and Godot, allowing developers to integrate local AI into their apps easily.

Who it’s for

Developers who want to integrate on-device AI capabilities into their applications across multiple platforms without relying on cloud-based LLM providers.

Highlights

  • Broad Platform Support: Works across iOS, Android, macOS, Windows, Linux, visionOS, and watchOS.
  • GGUF Compatibility: Supports any model in the GGUF format, with the ability to load models directly from Hugging Face or URLs.
  • Multimodal Capabilities: Supports text, image, and audio input, as well as local text-to-speech (Kokoro, Pocket TTS, Supertonic) and speech-to-text (Whisper).
  • Type-Safe Tool Calling: Automatically generates structured grammars from function signatures to enable tool calling without manual schema writing.
  • Hardware Optimized: Offloads layers to VRAM where available and falls back to CPU to maximize performance based on available memory.

Related

  • Project
    orailnoor/cross-platform-llm-client

    A cross-platform AI chat client that unifies local on-device LLM inference via llama.cpp with cloud API access for a flexible, privacy-focused chat experience.

  • Project
    mlc-ai/web-llm

    A high-performance in-browser LLM inference engine that uses WebGPU to run language models locally without server-side processing.

  • Project
    omnimind-ai/OmniInfer

    OmniInfer is a cross‑platform inference engine that lets you run LLMs and vision‑language models locally via a simple CLI and OpenAI‑compatible HTTP server. It supports many backends (llama.cpp, MLX, MNN, etc.), works on desktop, Android and iOS, and provides hardware‑aware optimisation and a terminal UI for chatting.

  • Project
    ggml-org/llama.cpp

    A plain C/C++ implementation for LLM and VLM inference with minimal setup and state-of-the-art performance across a wide range of hardware.

  • Project
    a-ghorbani/pocketpal-ai

    PocketPal AI is a free, open‑source mobile app that runs quantized LLMs and neural TTS entirely on‑device (iOS & Android). It lets you chat, hear the assistant speak, and use built‑in tools (calculator, date‑time, HTML rendering) without any cloud connection. Models are downloaded once from Hugging Face and executed via llama.cpp (CPU/GPU/NPU). Users can create custom “Pals” or download community assistants from PalsHub. Developers can extend the app by adding new Talents, TTS engines, or locales; the codebase is a standard React‑Native project with native bridges to llama.cpp and ONNX Runtime. MIT‑licensed, privacy‑first, and fully offline.