microsoft/onnxruntime-extensions

onnxruntime-extensions: A specialized pre- and post- processing library for ONNX Runtime

What it solves

It provides a way to include common pre- and post-processing steps directly into ONNX models, reducing the need for separate manual processing code in different languages across different platforms.

How it works

The library implements a set of custom operators for ONNX Runtime via Custom Operator ABIs. It allows users to convert Huggingface transformer data processing classes into ONNX processing graphs, which can then be merged with the main model for a unified inference session.

Who it’s for

Developers who use ONNX Runtime for model inference across various platforms (Windows, Linux, macOS, Android, iOS, and Web-Assembly) and languages (Python, Java, C#, C++).

Highlights

  • Custom operators for vision, text, and NLP models.
  • Support for multiple languages including Python, Java, C#, and C++.
  • Ability to generate processing models from Huggingface transformer classes.
  • Cross-platform compatibility including mobile and Web-Assembly.

Related

  • Project
    onnx/optimizer

    A C++ library for performing arbitrary optimizations on ONNX models, providing shared optimization passes to be reused across different backend implementations.

  • Project
    huggingface/transformers.js

    A JavaScript library that enables running pretrained machine learning models directly in the browser using ONNX Runtime, removing the need for a server.

  • Project
    onnxsim/onnxsim

    A tool to simplify ONNX models by performing constant folding and graph optimization to remove redundant operators and reduce complexity.

  • Project
    onnx/onnx-mlir

    A compiler that transforms ONNX graphs into optimized, low-runtime code using LLVM/MLIR technology for efficient model deployment.

  • Dispatch
    Convert Transformers to ONNX with Hugging Face Optimum

    Hugging Face provides three methods to convert Transformers models to ONNX, ranging from the low-level torch.onnx API to the high-level Optimum library.