Hugging Face Optimum for Graphcore IPU Integration

Hugging Face has partnered with Graphcore to integrate the Optimum library with Graphcore Intelligence Processing Units (IPUs), enabling the acceleration of Transformer models on hardware specifically designed for AI workloads. This integration allows developers to reduce prediction latency and optimize the training and fine-tuning of state-of-the-art models with minimal code changes.

Optimized BERT for IPU

BERT is the first model optimized for IPU through this partnership. Graphcore engineers have implemented and optimized BERT using Hugging Face Transformers to simplify the process of training and accelerating the model on IPU systems. Hugging Face intends to expand this support to include models for vision, speech, translation, and text generation in the coming months.

Technical Setup and Environment

To leverage Optimum on Graphcore hardware, users must configure a specific software stack involving the Poplar SDK and PopTorch.

Poplar SDK and PopTorch Configuration

Users must first set up the Poplar SDK environment by running enable scripts for Poplar and PopART (Poplar Advanced Runtime). For systems running Poplar SDK version 2.3 on Ubuntu 18.04, the environment variables are typically set within the /opt/gc/ directory.

PopTorch, a component of the Poplar SDK, allows PyTorch models to run on the IPU. A dedicated virtual environment is required to install the PopTorch wheel file corresponding to the SDK version.

Installing Optimum Graphcore

The optimum[graphcore] package serves as the interface between the Hugging Face Transformers library and Graphcore IPUs. This package must be installed within the activated PopTorch virtual environment.

Fine-Tuning BERT on SQuAD 1.1

Optimum provides a streamlined workflow for fine-tuning BERT on the SQuAD 1.1 dataset using the run_qa.py script. This process requires a fast tokenizer (backed by the Hugging Face Tokenizers library) and an ipu_config.json file to configure the IPU hardware.

Training Workflow

Training is facilitated by the IPUTrainer class in Optimum, which allows models to leverage the Graphcore hardware and software stack. The process involves loading a pretrained model (such as bert-base-uncased) and using the Hugging Face Datasets library to manage the SQuAD v1.1 training and validation files.

Performance Results

Fine-tuning the BERT model over 3 epochs with a batch size of 2 per device and a learning rate of 6e-5 yielded the following results:

  • Training Metrics:
    • Train Loss: 0.9465060763888888
    • Train Runtime: 368.4015
    • Train Samples: 88,524
    • Train Samples per Second: 720.877
    • Train Steps per Second: 2.809
  • Validation Metrics:
    • Exact Match: 80.6623
    • F1 Score: 88.2757
    • Eval Samples: 10,784

Sources