Convert Transformers to ONNX with Hugging Face Optimum

Hugging Face offers three distinct pathways to convert Transformers models to the Open Neural Network Exchange (ONNX) format, allowing users to choose between granular control and high-level abstraction. The most streamlined method is via the Optimum library, which automates the conversion process while maintaining compatibility with Hugging Face pipelines.

High-Level Conversion with Hugging Face Optimum

The Optimum library provides the most user-friendly method for ONNX conversion through ORTModelForXxx classes. By setting the from_transformers=True flag in the from_pretrained() method, Optimum automatically loads a vanilla Transformers model and converts it to ONNX using the transformers.onnx package internally.

Implementation Example:

from optimum.onnxruntime import ORTModelForSequenceClassification

model = ORTModelForSequenceClassification.from_pretrained("distilbert-base-uncased-finetuned-sst-2-english", from_transformers=True)

Models converted via Optimum can be used immediately for predictions or integrated directly into Hugging Face pipelines.

Mid-Level Conversion with transformers.onnx

The transformers.onnx module simplifies the conversion process by utilizing configuration objects, removing the need for users to manually define complex parameters such as dynamic_axes.

Implementation Example:

from pathlib import Path
import transformers
from transformers.onnx import FeaturesManager
from transformers import AutoConfig, AutoTokenizer, AutoModelForSequenceClassification

model_id = "distilbert-base-uncased-sst-2-english"
feature = "sequence-classification"
model = AutoModelForSequenceClassification.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

model_kind, model_onnx_config = FeaturesManager.check_supported_model_or_raise(model, feature=feature)

onnx_inputs,
    preprocessor=tokenizer,
    model=model,
    config=onnx_config,
    opset=13,
    output=Path("trfs-model.onnx")
)

Low-Level Conversion with torch.onnx

The torch.onnx API provides the most granular control but requires the manual specification of several parameters, including input_names, output_names, and dynamic_axes.

Implementation Example:

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "distilbert-base-uncased-finetuned-sst-2-english"
model = AutoModelForSequenceClassification.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)
dummy_model_input = tokenizer("This is a sample", return_tensors="pt")

torch.onnx.export(
    model, 
    tuple(dummy_model_input.values()),
    f="torch-model.onnx",  
    input_names=['input_ids', 'attention_mask'], 
    output_names=['logits'], 
    dynamic_axes={'input_ids': {0: 'batch_size', 1: 'sequence'}, 
                  'attention_mask': {0: 'batch_size', 1: 'sequence'}, 
                  'logits': {0: 'batch_size', 1: 'sequence'}}, 
    do_constant_folding=True, 
    opset_version=13,
)

Sources