orailnoor/cross-platform-llm-client

A unified cross-platform AI client supporting seamless transitions between standard cloud APIs and on-device, offline execution of custom and uncensored language models.

What it solves

PrivateLM provides a unified, cross-platform interface for interacting with Large Language Models (LLMs), solving the problem of choosing between privacy-focused local execution and the power of cloud-based APIs. It allows users to run models entirely on-device for privacy and offline use, while offering a seamless fallback to major cloud providers when higher performance is needed.

How it works

Built with Flutter, the app uses a dual-inference pipeline:

  • Local Inference: On Android and iOS, it wraps llama.cpp to run GGUF models. It automatically detects device hardware (RAM and GPU) to optimize thread counts and Vulkan/Metal GPU acceleration.
  • Cloud Inference: A normalization service connects to OpenAI, Anthropic, Google Gemini, and Kimi (Moonshot AI) APIs, allowing users to switch providers within a single UI.
  • Data Management: All chat history, settings, and API keys are stored locally on the device using Hive, ensuring that data only leaves the device when cloud mode is explicitly used.

Who it’s for

It is designed for users who want a single AI chat client that supports both private, offline local models and high-performance cloud LLMs across Android, iOS, and Web platforms.

Highlights

  • On-Device Execution: Runs GGUF models locally on Android (Vulkan) and iOS (Metal).
  • Hybrid Model Support: Seamlessly switches between local inference and cloud APIs (OpenAI, Claude, Gemini, Kimi).
  • Multimodal Capabilities: Supports text and image inputs, including vision support via local models like Qwen2-VL.
  • Hardware Optimization: Automatically recommends context size and token limits based on detected device RAM.
  • Local-First Privacy: Stores all sessions and configurations locally via Hive.

Related

  • Project
    lollipopkit/flutter_gpt_box

    A cross-platform GPT client for OpenAI and other AI APIs that provides a unified interface for multiple LLM providers with encrypted sync and advanced tools.

  • Project
    AtomicBot-ai/Atomic-Chat

    Atomic Chat is a desktop/mobile app that runs open‑weight LLMs locally and exposes them via an OpenAI‑compatible HTTP server (`http://localhost:1337/v1`). It supports multiple back‑ends (llama.cpp, a TurboQuant‑enhanced fork, and Apple‑Silicon MLX‑VLM), speculative decoding tricks for speed, cloud‑model fallbacks, and one‑click integration with many open‑source coding agents. The UI handles model download, engine selection, and API routing, making private on‑device inference easy for developers and researchers.

  • Project
    netdur/llama_cpp_dart

    A Dart FFI binding for llama.cpp that enables running LLMs, vision, and audio models locally within Flutter apps on iOS and Android.

  • Project
    Mobile-Artificial-Intelligence/maid

    Maid is an open‑source Android app that lets you run LLMs locally with llama.cpp (GGUF files) or connect to many cloud inference providers, offering model download, chat management, parameter tuning, optional Supabase sync, and a companion TTS app.

  • Project
    ImDarkTom/LlamaPen

    A web-based GUI for Ollama and other local LLM providers that allows users to chat with self-hosted models via a privacy-focused interface.