Groq Integration with Hugging Face Inference Providers

Hugging Face has integrated Groq as a supported Inference Provider on the Hugging Face Hub. This integration allows developers to leverage Groq's Language Processing Unit (LPU’s) high-speed inference for a variety of open-source models, including Meta's Llama 4 and Qwen's QWQ-32B, directly through the Hub's model pages and client SDKs.

High-Performance Inference via Groq LPU

Groq utilizes a specialized Language Processing Unit (LPU’s) designed specifically for computationally intensive applications with sequential components, such as Large Language Models (LLMs). By overcoming the limitations of traditional GPUs for inference, the LPU architecture provides significantly lower latency and higher throughput, making it optimized for real-time AI applications.

Groq provides an on-demand, pay-as-you-go API for accessing these openly-available models, which is now accessible as an Inference Provider on the Hugging Face Hub.

Integration and Workflow

Users can access Groq's inference capabilities through two primary interfaces: the Hugging Face website UI and the client SDKs.

Website UI Configuration

In the user account settings, users can manage their inference experience by:

  • Setting API Keys: Users can input their own API keys for specific providers. If no custom key is provided, requests are routed through Hugging Face.
  • Ordering Preferences: Users can set their preferred order of providers, which determines how they appear in the model page widgets and code snippets.

Client SDK Implementation

Groq is integrated into the Hugging Face client SDKs for both Python and JavaScript.

For Python users, the huggingface_hub library (version v0.33.0 or source installation) allows the use of the InferenceClient by specifying provider="groq".

For JavaScript users, the @huggingface/inference library allows the same functionality by passing the provider: "groq" parameter within the chatCompletion method.

Routing and Billing Models

There are two distinct modes for calling Inference Providers on the Hub:

  1. Custom Key Mode: Requests are sent directly to the inference provider using the user's own API key. In this mode, the user is billed directly by the provider (e.g., Groq).
  2. Routed by HF Mode: Requests are authenticated via the Hugging Face Hub. In this mode, the user does not need a provider-specific token, and charges are applied to the Hugging Face account. Hugging Face passes through the provider costs directly without adding a markup.

Subscription Benefits

  • PRO Users: Hugging Face PRO subscribers receive $2 worth of Inference credits every month, which can be used across different providers.
  • Free Users: Signed-in free users are provided with a small quota for free inference.

Sources