Hugging Face Inference Providers Integration

Hugging Face has integrated four serverless inference providers—fal, Replicate, SambaNova, and Together AI—directly into the Hub's model pages and client SDKs. This move transitions the Hub from hosting its own serverless Inference API to offering unified access to a diverse ecosystem of specialized serverless providers.

Unified Serverless Inference Access

Users can now access serverless inference for a wide variety of models directly from the Hub. This integration allows for easier exploration and prototyping of models like DeepSeek-R1 and FLUX.1-dev without needing to manage separate infrastructure.

Website UI Integration

In the Hub's user account settings, users can manage their inference experience through two primary methods:

  • Custom API Keys: Users can set their own API keys for specific providers. Requests using these keys go directly to the provider.
  • Provider Preference: Users can order providers by preference, which dictates how they appear in model page widgets and code snippets.

Routing Modes

There are two distinct modes for calling Inference APIs through this system:

  1. Custom Key Mode: Calls are sent directly to the inference provider using the user's own API key for that provider.
  2. Routed by HF: Requests are routed through Hugging Face. In this mode, users do not need a provider-specific token, and charges are applied directly to the Hugging Face account.

Technical Implementation and SDKs

Python Integration

Using the huggingface_hub library (v0.28.0 or later), developers can specify a provider in the InferenceClient:

from huggingface_hub import InferenceClient

client = InferenceClient(
    provider="together",
    api_key="xxxxxxxxxxxxxxxxxxxxxxxx"
)

completion = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1", 
    messages=[{"role": "user", "content": "What is the capital of France?"}], 
    max_tokens=500
)

JavaScript Integration

Developers using @huggingface/inference can pass the provider argument within the chatCompletion method:

import { HfInference } from "@huggingface/inference";

const client = new HfInference("xxxxxxxxxxxxxxxxxxxxxxxx");

const chatCompletion = await client.chatCompletion({
    model: "deepseek-ai/DeepSeek-R1",
    provider: "together",
    max_tokens: 500
});

HTTP Routing Proxy

Hugging Face exposes a routing proxy under the huggingface.co domain, enabling OpenAI-compatible API calls via the URL structure: https://router.huggingface.co/{provider}.

Billing and Credits

Billing depends on the chosen routing mode:

  • Direct Requests: Users are billed by the inference provider directly using their provider account.
  • Routed Requests: Users pay standard provider API rates passed through by Hugging Face with no additional markup.

Hugging Face PRO users receive $2 worth of Inference credits every month to use across providers. Signed-in free users are provided with a small quota of free inference.

Community Feedback and Perspectives

Community discussion on the announcement has highlighted several points of contention and technical hurdles:

"The previous pro provided 20,000 requests. Now it's gone. What a pity."

"I think this is bad decision. I have pay $9 and only use $2. I had 20k daily limit before."

Users have reported issues with InferenceClient.__init__() receiving unexpected keyword arguments for those not updated to the latest library version, as well as 403 Forbidden errors regarding permissions when calling Inference Providers on behalf of a user.

Sources