Hugging Face and Cloudflare Workers AI Integration

Hugging Face and Cloudflare have launched the "Deploy on Cloudflare Workers AI" integration, enabling developers to access popular open models as serverless APIs. This integration removes the need for developers to manage GPU infrastructure or pay for idle capacity by utilizing Cloudflare's global network of serverless GPUs.

Note: As of November 2024, this integration is no longer available. Users are advised to switch to the Hugging Face Inference API, Inference Endpoints, or other deployment options.

Serverless GPU Inference for Developers

Cloudflare Workers AI provides a low-cost, serverless solution to the common challenges of GPU scarcity and the fixed costs associated with deploying servers. By utilizing a pay-per-request pricing model, developers only pay for the compute they actually use.

To illustrate the cost-efficiency, Hugging Face provides an example of a Retrieval-Augmented Generation (RAG) application receiving approximately 1,000 requests per day (with 1,000 input tokens and 100 output tokens) using Meta Llama 2 7B; such an application would cost approximately $1 per day.

Technical Implementation and Workflow

Developers can deploy Hugging Face models on Cloudflare Workers AI through a streamlined process integrated directly into the Hugging Face Hub.

Deployment Process

  1. Model Selection: Users can find supported models—including Llama, Gemma, and Mistral—within a curated Cloudflare Collection on the Hugging Face Hub.
  2. Deployment Trigger: On the model page, users select the "Deploy" menu and choose "Cloudflare Workers AI."
  3. Configuration: The interface provides specific instructions and code snippets for the selected model.

Integration Options

Once deployed, the integration can be accessed via two primary methods:

  • Workers AI REST API: Requires the definition of ACCOUNT_ID and API_TOKEN variables.
  • Cloudflare AI SDK: Allows for direct integration within Cloudflare Workers.

Infrastructure and Model Support

The integration is powered by production solutions such as Hugging Face's Text Generation Inference (TGI). The models are hosted on state-of-the-art GPUs deployed across Cloudflare's edge data centers. While the initial launch included popular open models, the availability of specific models is determined by support within the Cloudflare Workers AI ecosystem; models without the "Cloudflare Workers AI" option in the deploy menu are not currently supported.

Sources