google-ai-edge/litert-samples

LiteRT and LiteRT-LM sample apps, model recipes, agent skills and utilities.

What it solves

It provides a comprehensive collection of reference implementations, model recipes, and deployment tools for developers using LiteRT (formerly TensorFlow Lite) and LiteRT-LM. It bridges the gap between having a raw machine learning model and deploying it efficiently on-device (Android, iOS, Web) using hardware acceleration like GPUs and NPUs.

How it works

The repository is organized into four main functional areas:

  • Application Samples: Runnable apps demonstrating various APIs, including the modern CompiledModel API for hardware acceleration, the legacy Interpreter API, and specialized LLM orchestration via LiteRT-LM.
  • Model Recipes: Conversion scripts and "cookbooks" that guide users through transforming models from platforms like Hugging Face into optimized .litertlm or .tflite formats.
  • Utilities: Toolkits for GPU optimization, such as patching PyTorch patterns to be compatible with LiteRT GPU delegates.
  • Agent Skills: Automated workflows that handle the model lifecycle, including conversion, quantization, on-device verification, and app scaffolding.

Who it’s for

Mobile and web developers, as well as ML engineers, who want to deploy high-performance AI models (vision, audio, and LLMs) directly on end-user devices.

Highlights

  • Diverse Modality Support: Includes samples for streaming text-to-speech (KittenTTS), automatic speech recognition (ASR), text-to-image diffusion (Bonsai Image), and multimodal apps (PhotoTalk).
  • Hardware Acceleration: Specific focus on maximizing performance via GPU/NPU acceleration and zero-copy buffers.
  • End-to-End Pipelines: Provides full paths from Hugging Face checkpoints to verified on-device deployment.
  • Interactive Playground: A Web/WASM-based environment to test Tensor API capabilities directly in the browser.

Related

  • Project
    google-ai-edge/LiteRT

    LiteRT is Google’s high‑performance on‑device runtime for running ML and generative‑AI models across phones, laptops, browsers, and IoT devices. It supersedes TensorFlow Lite with a unified Compiled Model API that automatically selects the best accelerator (CPU, GPU, NPU, WebGPU). The ecosystem includes converters for PyTorch, a CLI for coding‑agent workflows, a Web‑focused LiteRT.js, and a library for on‑device LLM inference. Supported platforms span Android, iOS, Linux, macOS, Windows, Web, and Raspberry Pi, with broad hardware acceleration support.

  • Project
    qualcomm/ai-hub-apps

    Qualcomm AI Hub Apps provides sample Android, Windows, and Ubuntu applications plus tutorials that demonstrate how to run vision, audio, and language models on Snapdragon devices using TensorFlow Lite, ONNX, and Qualcomm’s Genie SDK, with optional CPU/GPU/NPU acceleration.

  • Project
    google-ai-edge/LiteRT-LM

    A production-ready orchestration layer for running high-performance, cross-platform LLMs on-device using hardware acceleration.

  • Project
    Liquid4All/cookbook

    A comprehensive collection of examples, tutorials, and applications for deploying and fine-tuning Liquid AI's open-weight Liquid Foundation Models (LFMs) and LEAP SDK on mobile, desktop, and edge devices.

  • Project
    qualcomm/ai-hub-models

    Qualcomm AI Hub Models is a Python package and CLI that provides a catalog of pre‑optimised machine‑learning models (vision, language, generative AI) ready for deployment on Snapdragon CPUs, GPUs, and NPUs. It handles quantisation, compilation, and performance profiling on real Qualcomm devices via the AI Hub Workbench, and includes sample apps and demos for both local PyTorch inference and on‑device accelerated runtimes.