Hugging Face and Graphcore Partnership for IPU-Optimized Transformers

Hugging Face and Graphcore have partnered to integrate Graphcore's Intelligence Processing Units (IPUs) into the Hugging Face ecosystem. This collaboration, announced at the 2021 AI Hardware Summit, allows developers to deploy state-of-the-art Transformer models optimized for IPUs at production scale with minimal coding complexity.

Intelligence Processing Unit (IPU) Architecture

The Intelligence Processing Unit (IPU) is a processor specifically designed for the computational requirements of AI and machine learning, powering Graphcore’s IPU-POD datacenter compute systems. Unlike GPUs, which utilize a SIMD/SIMT architecture, the IPU employs a massively parallel MIMD architecture.

Key technical characteristics of the IPU include:

  • On-die Memory: Ultra-high bandwidth memory is placed adjacent to the processor cores directly on the silicon die.
  • Specialized Silicon: The hardware is built to handle sparsity, low precision arithmetic, and fine-grained parallelism.
  • Software Integration: The Poplar SDK, co-designed with the processor, integrates with standard frameworks including PyTorch and TensorFlow, as well as deployment tools like Docker and Kubernetes.

Optimizing Transformers via the Hardware Partner Program

To bridge the gap between advanced hardware and model deployment, Hugging Face launched the Hardware Partner Program. This initiative connects the Transformers library with optimized AI hardware to reduce the friction of adopting new compute platforms.

The Optimum Library

Developers will access hardware-optimized models through Optimum, a new open-source library and toolkit. This toolkit provides models certified by Hugging Face that are developed in collaboration with Graphcore. These IPU-optimized models are designed to be "plug-and-play," removing the need for developers to learn new code or processes to leverage the hardware.

Scope of Application

The partnership aims to cover a wide range of applications, including:

  • Natural Language Processing (NLP) such as text generation, sentiment analysis, and translation.
  • Computer Vision (e.g., ViT).
  • Speech processing.

Performance and Benchmarks

Graphcore has demonstrated the ability of the IPU to accelerate Transformer models using a specialized Graphcore-optimized implementation of Hugging Face BERT via PyTorch. According to Graphcore, the benchmark results for BERT running on an IPU system show dramatic performance advantages compared to comparable GPU-based systems, which can significantly reduce training time and increase the number of iterations possible during model development.

Sources