RunanywhereAI/runanywhere-sdks

RunAnywhere SDKs – On‑device AI for every platform

What it is – RunAnywhere provides a single C++ core that can run many AI capabilities (LLMs, vision‑language models, speech‑to‑text, text‑to‑speech, embeddings, RAG, image generation, tool‑calling, etc.) entirely on the device. Eight language‑specific SDKs (Swift, Kotlin, Flutter, React‑Native, Web, Electron, Python, and the wally CLI) expose the same high‑level API, while a capability registry picks the best hardware‑accelerated engine for the current device (Snapdragon Hexagon NPU, Apple MLX, llama.cpp, ONNX, Core ML, CUDA, WebGPU, or plain CPU).

Core ideas

Concept	Explanation
One API, many back‑ends	Call `RunAnywhere.capabilities()` to see what the current device can do. The runtime automatically routes each request to the highest‑priority engine that supports it.
On‑device, private by default	Models are downloaded on first use and run locally; no network traffic is required unless you deliberately start the optional OpenAI‑compatible server.
Hardware‑aware acceleration	• QHexRT – Snapdragon Hexagon NPU • MLX – Apple silicon (Metal/MLX) • llama.cpp – CPU, Metal, CUDA, WebGPU/WASM • sherpa + ONNX – speech, embeddings, etc. • Core ML – diffusion on iOS/macOS
Cross‑platform SDKs	Swift (iOS/macOS), Kotlin (Android), Flutter, React‑Native, Web (TS + WASM), Electron (Windows), Python, and the `wally` terminal tool. All SDKs share the same version line (e.g., 0.20.29).
Connect (LAN host/client)	macOS can host a loaded model; iOS, iPadOS, and Android can stream generations over a trusted local network (Bonjour discovery, plain TCP).

What you can build (on‑device)

LLM chat – dozens of models (Llama, Qwen, Gemma, Phi, SmolLM, etc.) with streaming, multi‑turn history, LoRA adapters.
Structured JSON output – schema‑validated generation, constrained decoding.
Tool calling – local function calls with stable IDs, parallel execution when supported.
Vision‑language (VLM) – image captioning, live camera description, photo Q&A.
Computer‑use action parser – parses Fara‑style action strings into screen‑scaled coordinates.
Speech‑to‑Text – Whisper, Moonshine, batch or live streams.
Text‑to‑Speech – neural voices (Piper, Kokoro, Kitten, MeloTTS, Magpie).
Voice agents – pipeline of VAD → STT → LLM → TTS (no wake‑word detection).
Embeddings & RAG – local document ingestion, vector search, streaming answers.
Image generation – Stable Diffusion on Core ML, in‑painting on Hexagon NPU.

Quick start examples

Python (pip)

pip install runanywhere==0.20.11

import runanywhere as ra
from runanywhere import LlmOptions

ra.initialize()  # downloads model on first call
for ev in ra.llm.generate_stream("Explain on‑device AI in one sentence.",
                                 LlmOptions(model="qwen2.5-0.5b")):
    if ev.is_token:
        print(ev.text, end="")

Swift (iOS/macOS)

import RunAnywhere
import LlamaCPPRuntime

LlamaCPP.register()
try RunAnywhere.initialize()

var load = RAModelLoadRequest()
load.modelID = "smollm2-360m"
load.category = .language
load.framework = .llamaCpp
await RunAnywhere.loadModel(load)

var req = RALLMGenerateRequest()
req.prompt = "What is the capital of France?"
let result = try await RunAnywhere.generate(req)
print(result.text)   // → Paris

Web (TypeScript, WASM + WebGPU)

npm i @runanywhere/web@0.20.11 @runanywhere/web-llamacpp@0.20.11

import { RunAnywhere } from "@runanywhere/web";
import { LlamaCPP } from "@runanywhere/web-llamacpp";
await RunAnywhere.initialize({ environment: "DEV" });
await LlamaCPP.register({ acceleration: "auto" });
await RunAnywhere.loadModel({ modelId: "qwen2.5-0.5b" });
const out = await RunAnywhere.generate({ prompt: "Capital of France?" });
console.log(out.text); // Paris

(Analogous snippets are provided for Kotlin, Flutter, React‑Native, Electron, and the wally CLI.)

SDK matrix (status & install)

SDK	Platforms	Status	Install command
Swift	iOS 17.5+, macOS 14.5+	Stable	Swift Package Manager (`https://github.com/RunanywhereAI/runanywhere-sdks`)
Kotlin	Android API 24+	Stable	Gradle: `implementation("io.github.sanchitmonga22:runanywhere-sdk:0.20.11")`
Flutter	iOS, Android	Beta	`pub add runanywhere` (plus backend packages)
React Native	iOS, Android	Beta	`npm i @runanywhere/core @runanywhere/llamacpp`
Web	Chromium, Safari, Firefox	Beta	`npm i @runanywhere/web @runanywhere/web-llamacpp`
Electron	Windows x64	Preview	Build from source (see `bindings/electron/`)
Python	Windows/macOS/Linux	Alpha	`pip install runanywhere`
wally (CLI)	macOS, Linux, Windows	Stable	`curl …

How the runtime chooses an engine

Each engine registers the modalities it can run and the hardware it targets.
When a request arrives, the capability registry looks at the device’s available hardware and picks the highest‑priority engine that satisfies the request.
If no engine matches, the call fails with a clear error; you can query capabilities() to see what is actually installed.

Typical priority order (high → low):

QHexRT (Snapdragon NPU) → MLX (Apple silicon) → llama.cpp (GPU/CPU) → sherpa + ONNX → Core ML → fallback CPU.

Where to learn more

Docs: https://docs.runanywhere.ai (platform‑specific guides)
Model catalog: https://huggingface.co/runanywhere/models
Discord community: https://discord.gg/N359FBbDVd
GitHub releases: https://github.com/RunanywhereAI/runanywhere-sdks/releases

TL;DR

RunAnywhere is a genuine, production‑grade SDK suite that lets developers write one piece of code and run a wide range of AI models locally on phones, browsers, desktops, or servers, automatically leveraging the best hardware accelerator available. It covers LLM chat, vision‑language, speech, embeddings, RAG, and diffusion, with stable APIs for Swift, Kotlin, Flutter, React‑Native, Web, Electron, Python, and a CLI.