Amazon SageMaker and Hugging Face Partnership

TL;DR

Hugging Face and Amazon have entered a strategic partnership to simplify the training and deployment of state-of-the-art NLP models by integrating Hugging Face Transformers into Amazon SageMaker. This collaboration introduces dedicated Hugging Face Deep Learning Containers (DLCs) and a first-class extension for the SageMaker Python SDK, reducing the time to set up and run experiments from days to minutes.

Hugging Face Deep Learning Containers (DLCs)

Amazon SageMaker now provides Hugging Face DLCs, which are pre-installed Docker images containing deep learning frameworks and libraries such as transformers, datasets, and tokenizers. These containers eliminate the need for users to build and optimize environments from scratch.

Key characteristics of the DLCs include:

  • Framework Support: Optimized variants are available for both TensorFlow and PyTorch.
  • Infrastructure Flexibility: Support for single-GPU, single-node multi-GPU, and multi-node clusters.
  • Performance: Built-in optimizations for PyTorch and TensorFlow to increase training speed and optimize the price/performance ratio.
  • Open Source: The DLCs are licensed under Apache 2.0.

SageMaker Python SDK Integration

To accelerate the transition from data science to production, Hugging Face developed a first-class extension for the SageMaker Python SDK. This integration allows users to create a HuggingFace Estimator to manage the end-to-end training process.

Core Capabilities

  • Automatic Model Tuning: Integration with SageMaker's Automatic Model Tuning to optimize hyperparameters and increase model accuracy.
  • Experiment Tracking: Compatibility with SageMaker Studio IDE for tracking and comparing training artifacts and experiments.
  • Managed Infrastructure: SageMaker automatically manages the Amazon EC2 instances, uploads the training script (e.g., train.py), and handles data I/O from Amazon S3.

Advanced Training Features

The partnership introduces several high-performance training capabilities to handle large-scale NLP workloads.

Distributed Training

Users can leverage SageMaker distributed training libraries directly through the HuggingFace Estimator by defining a distribution parameter:

  • Data Parallelism: Integrated into the Trainer API to simplify training on large datasets.
  • Model Parallelism: Supports training models with billions of parameters by partitioning the model across multiple GPUs/nodes.

Cost and Workflow Optimization

  • Managed Spot Instances: Users can utilize EC2 spot instances to reduce training costs by up to 90%. This is recommended in conjunction with checkpoint_s3_uri for long-running jobs.
  • Git Repository Integration: The entry_point and source_dir for the estimator can be pointed directly to a GitHub repository, removing the need to download scripts locally.
  • SageMaker Metrics: The system can automatically parse logs using regular expressions to send training metrics (e.g., eval_loss, eval_accuracy) to Amazon CloudWatch.

Deployment and Inference

While the initial focus was on training, the integration was extended on July 8, 2021, to include easy deployment and inference of Transformers models within Amazon SageMaker.

Summary of Benefits

According to the Hugging Face FAQ, using SageMaker for Hugging Face models provides three primary advantages:

  1. Cost-Effectiveness: Instances are paid per second and terminate automatically at the end of the job.
  2. MLOps Automation: Training metadata, logs, and S3 I/O for checkpoints and artifacts are fully managed.
  3. Scalability: Enables the launch of multiple parallel training jobs and large-scale distributed training.

Sources