pytorch/executorch

On-device AI across mobile, embedded and edge for PyTorch

What it solves

ExecuTorch provides a unified, PyTorch-native way to deploy AI models locally on edge devices. It eliminates the complexity of converting models to different formats for every single piece of hardware, allowing developers to run high-performance inference on everything from smartphones and laptops to wearables and microcontrollers.

How it works

ExecuTorch uses a torch.export workflow to capture a PyTorch model and then optimizes it for a specific target. This process creates a backend-specific .pte file that can be executed by a lightweight runtime. The system allows for partitioned hardware acceleration, where specific parts of the model graph are delegated to a GPU, NPU, or DSP, while keeping portable CPU kernels as a fallback.

Who it’s for

It is designed for AI engineers and embedded systems developers who need to run PyTorch models on-device without sacrificing performance or native PyTorch integration.

Highlights

  • Broad Hardware Support: Supports Android, iOS, macOS, Linux, Windows, WebAssembly, and embedded systems like Arm Cortex-M and Arduino.
  • PyTorch-Native: Works directly from torch.export, maintaining source mappings for easier debugging.
  • Right-Sized Runtime: Allows selective builds to include only the necessary operators and kernels, reducing the binary size for constrained devices.
  • Diverse Workload Support: Optimized for local LLMs, voice (speech recognition/synthesis), multimodal models, and computer vision.
  • Enterprise Scale: Powers on-device AI for Meta's apps like Instagram, WhatsApp, and Facebook.

Related

  • Project
    pytorch/torchtitan

    torchtitan is a PyTorch‑native library for large‑scale LLM pre‑training, fine‑tuning, and reinforcement‑learning, offering composable parallelism (FSDP, TP, PP, CP, EP), recipe‑based configuration, and support for frontier models and low‑precision formats.

  • Project
    NVIDIA/physicsnemo

    NVIDIA PhysicsNeMo is a PyTorch-based framework for building and scaling AI models for scientific and engineering simulations, supporting diverse data structures like meshes and grids.

  • Project
    software-mansion/react-native-executorch

    React Native ExecuTorch is a library that brings Meta’s on‑device ExecuTorch inference runtime to React Native, enabling offline AI (LLMs, vision, speech, etc.) with hardware acceleration and ready‑made hooks or low‑level TypeScript pipelines.

  • Project
    k2-fsa/sherpa

    An open-source speech-to-text inference framework using PyTorch for deploying pre-trained transducer and CTC-based models.

  • Project
    google-ai-edge/litert-torch

    A Python library for converting PyTorch models into .tflite format, enabling high-performance on-device execution on Android, iOS, and IoT devices.