Hugging Face and AMD Partnership for Model Acceleration

Hugging Face and AMD have entered a strategic partnership to accelerate the performance of state-of-the-art transformer models across AMD's CPU and GPU platforms. This collaboration aims to provide developers with more hardware options and better cost-performance standards for the pre-training, fine-tuning, and inference of large-scale models.

Supported Hardware Platforms

AMD and Hugging Face are optimizing performance across three primary hardware categories:

  • GPUs: The collaboration focuses first on enterprise-grade Instinct MI2xx and MI3xx families, followed by customer-grade Radeon Navi3x family. Initial testing indicates that the MI250 trains BERT-Large 1.2x faster and GPT2-Large 1.4x faster than its direct competitor.
  • CPUs: Optimization for inference will target both client Ryzen and server EPYC CPUs, leveraging model compression techniques like quantization to make CPUs a viable option for transformer inference.
  • AI Accelerators: The partnership includes the Alveo V70 AI accelerator, designed to provide high performance with lower power requirements.

Supported Model Architectures and Frameworks

The partnership intends to support a wide array of transformer architectures across natural language processing, computer vision, and speech, including:

  • Generative AI: GPT2, GPT-NeoX, T5, OPT, LLaMA, BLOOM, and StarCoder.
  • General Transformers: BERT, DistilBERT, ROBERTA, Vision Transformer, CLIP, and Wav2Vec2.
  • Other Architectures: Traditional computer vision models such as ResNet and ResNext, as well as deep learning recommendation models.

These models will be tested and validated for use with PyTorch, TensorFlow, and ONNX Runtime, although availability for training and inference may vary by framework and hardware platform.

Implementation and Integration Roadmap

To ensure a seamless user experience, Hugging Face and AMD are focusing on the following technical integrations:

  • ROCm SDK Integration: The AMD ROCm SDK will be integrated into Hugging Face's open-source libraries, starting with the transformers library.
  • Dedicated Optimum Library: The collaboration is expected to result in a new Optimum library specifically dedicated to AMD platforms, allowing users to leverage AMD hardware with minimal to no code changes.
  • Out-of-the-Box Optimization: The initial priority is ensuring that the most community-critical models work efficiently on AMD platforms by leveraging the latest AMD hardware and software features.

Sources