Getting Started with Transformers on Habana Gaudi

Hugging Face and Habana Labs have partnered to accelerate the training of Transformer models, utilizing Habana Gaudi accelerators to provide up to 40% better price-performance compared to the latest GPU-based Amazon EC2 instances.

Deploying Habana Gaudi on AWS

The most efficient way to access Habana Gaudi accelerators is through Amazon EC2 DL1 instances, which feature eight Habana Gaudi processors. To set up an environment, users can utilize the Habana Deep Learning Amazon Machine Image (AMI), which comes pre-installed with the Habana SynapseAI® SDK and the necessary tools for running Gaudi-accelerated Docker containers.

Instance Configuration

Setting up a Gaudi instance involves the following specifications:

  • Instance Type: dl1.24xlarge (the only available size).
  • Region: us-east-1.
  • Storage: A minimum of 50GB of Amazon EBS storage is recommended, as the default 8GB is insufficient for training tasks.
  • Cost Management: Utilizing Spot Instances can significantly reduce costs; for example, reducing the hourly rate from $13.11 to $3.93 (a 70% saving).

Software Environment

After launching the instance via SSH, users must pull a specific Habana Docker container for PyTorch to ensure compatibility. The recommended image for this setup is vault.habana.ai/gaudi-docker/1.5.0/ubuntu20.04/habanalabs/pytorch-installer-1.11.0:1.5.0-610.

Fine-Tuning Transformers with Optimum Habana

Training Transformer models on Habana Gaudi is facilitated by the optimum-habana library. This package allows users to leverage Gaudi hardware for tasks such as text classification.

Implementation Workflow

To fine-tune a model, the following steps are required:

  1. Install Optimum Habana: Clone the optimum-habana repository and install the package from source.
  2. Prepare Environment: Install the necessary Python dependencies via the requirements.txt file in the text-classification example directory.
  3. Execute Training: Use the run_glue.py script to launch the training job.

Performance Example: BERT Text Classification

In a demonstration fine-tuning a bert-large-uncased-whole-word-masking model on the MRPC task of the GLUE benchmark, the following results were achieved:

  • Training Time: 2 minutes and 12 seconds.
  • F1 Score: 0.9181.
  • Training Throughput: 82.824 samples per second.
  • Evaluation Accuracy: 0.8505.

Users can fetch Habana Gaudi configurations for BERT and other popular models directly from the Habana organization on the Hugging Face hub.

Sources