nobodywho-ooo/nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

What it solves

NobodyWho provides a way to run Large Language Models (LLMs) locally and offline on a wide variety of devices without requiring API keys, internet connectivity, or recurring fees.

How it works

It is an inference engine powered by llama.cpp that supports models in the GGUF format. It uses Vulkan or Metal for GPU acceleration across different operating systems and implements conversation-aware preemptive context shifting to maintain conversation memory without message length limits. The project provides bindings for multiple languages and frameworks, including Kotlin, Swift, Python, Flutter, React Native, and Godot.

Who it’s for

Developers building cross-platform applications (mobile, desktop, and game engines) who want to integrate private, on-device AI capabilities like chat, speech-to-text, and text-to-speech.

Highlights

  • Broad Platform Support: Native integrations for Kotlin, Swift, Python, Flutter, React Native, and Godot.
  • Multimodal Capabilities: Supports image and audio input, as well as speech-to-text (via Whisper) and text-to-speech (via Kokoro, Pocket TTS, and Supertonic).
  • Automated Tool Calling: Generates structured grammars from function signatures to enable type-safe tool calling without manual schema writing.
  • Flexible Model Loading: Allows loading models directly from Hugging Face or any custom URL.

Related

  • Project
    ggml-org/llama.cpp

    A high-performance C/C++ implementation for local LLM and VLM inference, supporting a wide range of hardware through quantization and hardware acceleration.

  • Project
    OpenBMB/MiniCPM-V-Apps

    A collection of on-device mobile demos for the MiniCPM-V family of multimodal models, enabling local AI inference on iOS, Android, and HarmonyOS.

  • Project
    ngxson/wllama

    A WebAssembly binding for llama.cpp that enables running LLMs, multimodal models, and tool calling directly in the web browser using WebGPU and Wasm.

  • Project
    foldl/chatllm.cpp

    A C++ LLM inference engine based on ggml that enables memory-efficient local execution of multimodal models with RAG and quantization support.

  • Project
    mlc-ai/web-llm

    A high-performance in-browser LLM inference engine that uses WebGPU to run language models locally without a server.