HUGS launch: zero‑configuration, hardware‑optimized inference for open LLMs

TL;DR

Hugging Face announced HUGS (Hugging Face Generative AI Services), a zero‑configuration, hardware‑optimized inference service for open‑source LLMs that runs on NVIDIA, AMD, and soon AWS Inferentia and Google TPUs, letting enterprises deploy models in‑house with an OpenAI‑compatible API.

Zero‑Configuration Optimized Inference for Open Models

HUGS removes the engineering burden of tuning inference for specific GPUs or AI accelerators. Each deployment is pre‑tested and automatically optimized for the target hardware, delivering maximum throughput without manual configuration. The service exposes an OpenAI‑compatible API, so existing applications can switch to HUGS simply by changing the endpoint URL.

Why HUGS?

HUGS is designed for enterprises that need to host open models securely and efficiently:

  • In‑house deployment – Models run inside the organization’s own infrastructure, keeping data and intellectual property off the public internet.
  • Zero‑configuration – Deployment time drops from weeks to minutes; HUGS automatically selects the optimal model and serving settings for NVIDIA, AMD, or other accelerators.
  • Hardware‑optimized – Built on Hugging Face’s Text Generation Inference (TGI), HUGS extracts peak performance from each supported accelerator.
  • Hardware flexibility – Currently supports NVIDIA and AMD GPUs; AWS Inferentia and Google TPUs are slated for future release.
  • Model flexibility – Works with a broad catalog of open‑source LLMs, giving developers freedom of choice.
  • Standard APIs – Deploy via Kubernetes with endpoints that mimic the OpenAI API, minimizing code changes.
  • Enterprise distribution – Includes long‑term support, rigorous testing, SOC2 compliance, and bundled licenses to reduce compliance risk.

"HUGS is a huge timesaver to deploy locally ready‑to‑work models with good performances – before HUGS it would take us a week, now we can be done in less than 1 hour. For customers with sovereign AI requirements it's a game changer!" – Henri Jouhaud, CTO at Polyconseil

"We tried HUGS to deploy Gemma 2 on GCP using an L4 GPU – we didn't have to fiddle with libraries, versions and parameters, it just worked out of the box. HUGS gives us confidence we can scale our internal usage of open models!" – Ghislain Putois, Research Engineer at Orange

How It Works

Where to Find HUGS

HUGS can be obtained through several distribution channels:

  1. Cloud Service Provider (CSP) Marketplaces – Deployable from the AWS Marketplace and Google Cloud Marketplace. Azure support is announced for the near future.
  2. DigitalOcean – Available as a 1‑Click Models service on GPU Droplets.
  3. Enterprise Hub – Accessible to organizations with an Enterprise Hub subscription via a sales request.

Each channel provides platform‑specific deployment instructions linked in the documentation.

Pricing

HUGS uses an on‑demand pricing model based on container uptime:

  • AWS & GCP Marketplaces – $1 per hour per container, with compute billed separately by the CSP. AWS offers a 5‑day free trial.
  • DigitalOcean – No additional HUGS charge; only the underlying GPU Droplet compute cost applies.
  • Enterprise Hub – Custom pricing; interested parties should contact Hugging Face sales.

Running Inference

HUGS leverages TGI and presents an OpenAI‑compatible Messages endpoint. Developers can call the service with standard tools such as curl, the huggingface_hub SDK, or the openai Python SDK. Example using huggingface_hub:

from huggingface_hub import InferenceClient

ENDPOINT_URL = "REPLACE"  # your deployed URL or IP
client = InferenceClient(base_url=ENDPOINT_URL, api_key="-")

chat_completion = client.chat.completions.create(
    messages=[{"role": "user", "content": "What is Deep Learning?"}],
    temperature=0.7,
    top_p=0.95,
    max_tokens=128,
)

Supported Models and Hardware

At launch, HUGS supports 13 popular open LLMs, spanning Meta Llama‑3.1, NousResearch, Mistral, Google Gemma‑2, and Qwen families. The service runs on NVIDIA and AMD GPUs, with upcoming support for AWS Inferentia and Google TPUs. Detailed compatibility matrices are maintained in the official Supported Models and Supported Hardware documentation.

Getting Started

Enterprises can begin using HUGS today by deploying from the AWS Marketplace, Google Cloud Marketplace, or DigitalOcean 1‑Click Models. The zero‑configuration nature of HUGS enables rapid proof‑of‑concept development and smooth migration from closed‑source APIs to self‑hosted open models.

Note (September 2025 update): HUGS model deployment containers are no longer offered. For optimized Hugging Face model deployment, see the Dell Enterprise Hub and the Hugging Face collection in Azure AI Foundry.

Sources