Hugging Face and Google Cloud Vertex AI Model Garden Integration

Hugging Face and Google Cloud have introduced "Deploy on Google Cloud," a new integration that allows developers to deploy thousands of open foundation models as API endpoints within their own Google Cloud accounts. This integration simplifies the transition from model discovery to production deployment by utilizing Vertex AI or Google Kubernetes Engine (GKE).

Simplified Deployment via Vertex AI and GKE

Developers can now deploy open models to production-ready endpoints without managing underlying infrastructure or servers. This is achieved through two primary entry points:

Deployment from the Hugging Face Hub

For models tagged with "text-generation-inference," users can select the "Deploy" menu on a model card and choose "Google Cloud." This action redirects the user to the Google Cloud Console, where they can deploy the model to Vertex AI with a single click or use manifest templates to deploy to a GKE Kubernetes cluster.

Deployment from Vertex Model Garden

Google Cloud users can discover and deploy models directly within the Google Cloud console using the "Deploy From Hugging Face" option in the Vertex Model Garden. This feature allows users to search for model IDs, access hundreds of popular open LLMs with pre-tested hardware configurations, and prefill required deployment settings for either Vertex AI or GKE.

Technical Infrastructure and Compatibility

The integration leverages Hugging Face's production solution, Text Generation Inference (TGI), to power inference for the most popular open models. To ensure security and authorization, users deploying gated models must provide their Hugging Face access token to authorize the model download.

Strategic Goal for AI Builders

The partnership aims to reduce the time and resources required to deploy open Generative AI models securely and reliably. By providing dedicated configurations and assets for Hugging Face models within the Vertex AI ecosystem, the integration removes the infrastructure management burden from developers building Generative AI applications.