Hugging Face and AWS Strategic Partnership for AI Accessibility

Hugging Face and Amazon Web Services (AWS) have expanded their long-term strategic partnership to accelerate the availability of next-generation machine learning (ML) models. This collaboration aims to make generative AI more accessible to the global developer community by reducing costs and improving performance through the integration of Hugging Face's model hub with AWS's cloud infrastructure.

Democratizing Generative AI Access

AWS and Hugging Face are partnering to provide the global AI community with access to next-generation models, countering the trend where popular generative AI models remain proprietary to large tech companies. The partnership focuses on making it faster and easier for developers to build, train, and deploy Transformer and Diffuser models that process and generate text, audio, and images.

To achieve this, Hugging Face has designated AWS as a preferred cloud provider. This allows the Hugging Face community to utilize state-of-the-art AWS tools for model optimization, including:

  • Amazon SageMaker: Used for building, training, and deploying models.
  • AWS Trainium: Purpose-built accelerators for training ML models.
  • AWS Inferentia: Purpose-built accelerators for inference.

Clement Delangue, CEO of Hugging Face, stated that accessibility and transparency are essential for sharing progress and using new capabilities responsibly, noting that "Amazon SageMaker and AWS-designed chips will enable our team and the larger machine learning community to convert the latest research into openly reproducible models that anyone can build on."

Scaling Machine Learning in the Cloud

The partnership enables the seamless integration of models hosted on the Hugging Face Hub—which hosts over 100,000 free and accessible models—with the capabilities of Amazon SageMaker and Amazon Elastic Computing Cloud (EC2).

Key technical and operational benefits include:

  • Simplified Deployment: Customers can fine-tune and deploy state-of-the-art Hugging Face models on Amazon SageMaker and Amazon EC2 with a few clicks.
  • Cost and Performance Optimization: By leveraging AWS Trainium and AWS Inferentia, developers can achieve higher performance at lower costs, lowering the barrier to entry for companies that previously found the cost and expertise required for generative AI prohibitive.

Adam Selipsky, CEO of AWS, highlighted that the partnership is designed to put innovative technology into the hands of more customers by making it easier to access popular ML models to create generative AI applications.

Sources