mil-tokyo/webdnn

The Fastest DNN Running Framework on Web Browser

What it solves

WebDNN allows developers to run neural network inference directly within a web browser, removing the need for a dedicated server to process AI models. It specifically simplifies the process by allowing ONNX models to be loaded directly into the browser without requiring Python preprocessing.

How it works

WebDNN converts and executes ONNX models using various hardware acceleration technologies available in the browser. It supports three primary backends:

  • WebGPU (using WGSL) for modern browsers like Chrome and Edge.
  • WebGL (with WebGL2 and WebGL1 fallback) for broader compatibility.
  • WebAssembly (via emscripten) for CPU-based execution.

Who it’s for

Web developers who want to integrate AI model inference into their web applications without relying on external APIs or server-side processing.

Highlights

  • Direct ONNX Support: Loads ONNX models straight into the browser without Python preprocessing.
  • Multiple Acceleration Backends: Supports WebGPU, WebGL, and WebAssembly for optimal performance.
  • Offline Optimization: Provides the system to optimize models offline before deployment.
  • Dynamic Loading: Operator bundles can be loaded dynamically at runtime to reduce initial load times.

Related

  • Project
    mlc-ai/web-llm

    A high-performance in-browser LLM inference engine that uses WebGPU to run language models locally without a server.

  • Project
    Tencent/ncnn

    A high-performance neural network inference framework optimized for mobile, embedded, and desktop deployment with no third-party runtime dependencies.

  • Project
    microsoft/onnxruntime-genai

    A high-performance API for running generative AI models on-device using ONNX Runtime, handling the complete generative loop from pre-processing to sampling.

  • Project
    microsoft/onnxruntime

    A cross-platform machine learning accelerator that optimizes inference and training for models from various frameworks like PyTorch and TensorFlow.

  • Project
    onnx/onnx-tensorrt

    A TensorRT backend for ONNX that parses ONNX models for high-performance execution on NVIDIA GPUs.