callstackincubator/ai

On-device LLM execution in React Native with Vercel AI SDK compatibility

What it solves

React Native AI solves the problem of running AI models directly on mobile devices without sending data to servers. This keeps user data private, reduces latency, and avoids server costs. It also makes it easy for React Native developers to use familiar AI SDK functions like generateText and streamText with local models.

How it works

The project provides a set of on-device AI primitives for React Native, with multiple providers that handle different hardware and model types. Some providers (Apple, Google ADK) use built-in system models, so no downloads are needed. Others (Llama, MLC) require downloading model files. All providers expose a common interface compatible with the Vercel AI SDK, so developers can write code that works with local or cloud models. For debugging, it includes a DevTools plugin that captures AI SDK telemetry and shows it in Rozenite DevTools.

Who it’s for

This is for React Native developers who want to add AI features like text generation, embeddings, transcription, speech synthesis, or image generation to their mobile apps, while keeping data on-device. It’s also useful for those who want to use the Vercel AI SDK with local models, or who need to run large language models on mobile devices with limited context. The project supports iOS and Android, with specific providers for each platform.

Highlights

  • Immediate use – Apple and Google ADK providers require no model downloads; they use system models built into the device.
  • Privacy-first – All inference runs on-device by default, and cloud models are opt-in (only for Google ADK).
  • Vercel AI SDK compatible – Use standard AI SDK functions like generateText, streamText, embed, and transcribe with local models.
  • Multiple providers – Choose from Apple Foundation Models, Core AI bundles, Google ADK (Gemini Nano), Llama via llama.rn, or MLC LLM.
  • Beyond text – Support for embeddings, transcription, speech synthesis, and image generation (Core AI).
  • JSON UI – The @react-native-ai/json-ui package lets a tool-calling model build React Native UI dynamically.
  • DevTools integration – The AI SDK Profiler shows telemetry from AI calls in Rozenite DevTools.

Related

  • Project
    EvanZhouDev/openai-oauth

    A tool and SDK that lets you use your ChatGPT account as an OpenAI-compatible API, enabling developers to build apps where users bring their own AI accounts.

  • Project
    1CatAI/1Cat-vLLM

    1Cat‑LLM is a vLLM fork that optimises LLM inference for NVIDIA Volta (Tesla V100) GPUs, offering custom kernels, FP8/E4M3 KV‑cache, GGUF support and speculative decoding to achieve up to ~260 tokens / s on 27‑B models while reporting quality‑aware benchmarks.

  • Project
    NVIDIA/gdrcopy

    A low-latency GPU memory copy library based on NVIDIA GPUDirect RDMA that allows the CPU to directly map and manipulate GPU memory to reduce transfer overhead.

  • Project
    JuliaGPU/CUDA.jl

    CUDA.jl is a Julia package that provides a high‑level array abstraction, a Julia‑to‑CUDA kernel compiler, and wrappers for NVIDIA libraries, enabling GPU‑accelerated computing for AI, scientific, and data‑intensive workloads.