DenisovAV/flutter_edge_ai

On-device AI for Flutter — Gemma and other LLMs, multimodal, function calling, embeddings, RAG and speech on Android, iOS, Web, macOS, Windows and Linux.

What it solves

This project provides a modular framework for running AI models directly on mobile and desktop devices using Flutter, eliminating the need for cloud servers or internet connectivity. It enables developers to integrate text, vision, audio, and RAG capabilities into cross-platform applications while maintaining user privacy and reducing latency.

How it works

The framework uses a core registry system that allows developers to plug in specific runtime engines (such as LiteRT, ONNX, or built-in OS models like Gemini Nano or Apple Foundation Models). It is designed as a collection of opt-in packages, meaning developers only include the specific AI features—like vector stores, speech processing, or agent tools—that their application requires.

Who it’s for

It is for Flutter developers who want to build privacy-focused, offline-capable AI applications across Android, iOS, web, and desktop platforms.

Highlights

  • Supports multiple inference backends including LiteRT, ONNX Runtime, and native OS-level AI models.
  • Includes built-in support for on-device RAG using vector stores like SQLite and Qdrant.
  • Provides modular packages for speech-to-text, text-to-speech, and agentic skill execution.
  • Offers hybrid routing for Genkit, allowing seamless switching between local and cloud-based models.

Related

  • Project
    pymc-devs/pytensor

    A Python library for defining and optimizing symbolic mathematical expressions with multi-dimensional arrays, serving as the computational backend for PyMC.

  • Project
    zhizinan1997/jimeng-free-api-all

    An OpenAI-compatible API gateway for Jimeng AI, providing access to its image and video generation models with built-in account pooling and risk-control bypass.

  • Project
    arcships/light-ocr

    light‑ocr is a cross‑platform, offline OCR library for Node.js and C++. It bundles the model, PDF renderer, and a Chinese fallback font in a single npm package, provides pre‑built native binaries for macOS, Linux and Windows, and can accelerate inference with Core ML or WebGPU. The API returns text lines with confidence scores and quadrilateral coordinates, and a CLI offers quick command‑line usage. Model sizes range from a 6 MB Tiny tier to a 139 MB Medium tier, all under Apache 2.0.

  • Project
    aws-neuron/aws-neuron-sdk

    AWS Neuron is an SDK for high-performance deep learning and generative AI, enabling the deployment and profiling of workloads on AWS Inferentia and Trainium accelerators.