Hugging Face Inference Solutions Overview November 2022

TL;DR

Hugging Face introduced a range of inference services—free widget, free API, paid Inference Endpoints, and Spaces—so developers can test, prototype, and deploy models with minimal friction and cost‑effective scaling.

Free Inference Widget: Instant Model Exploration

The Inference Widget lives on every model page and lets users upload sample data and obtain predictions with a single click, without writing code. Models are loaded on‑demand on Hugging Face servers and unloaded when idle, keeping the service free. This widget provides a quick sanity check of model behavior, output format, and sample performance.

"It's the best way to quickly get a sense of what a model does, its output, and how it performs on a few samples from your dataset. The model is loaded on-demand on our servers and unloaded when it's not needed anymore. You don't have to write any code and the feature is free. What's not to love?" – Julien Simon

Free Inference API: One‑Line HTTP Calls for Development

The Inference API powers the widget and offers a simple HTTP endpoint. By sending a POST request with the model identifier, input payload, and a Hub token, developers can obtain predictions in seconds.

curl https://api-inference.huggingface.co/models/xlm-roberta-base \
    -X POST \
    -d '{"inputs": "The answer to the universe is <mask>."}' \
    -H "Authorization: Bearer HF_TOKEN"

Key characteristics:

  • No custom server or container required.
  • Immediate model swapping for rapid comparison.
  • Free usage tier, subject to rate limiting.
  • Not recommended for production workloads due to rate limits and lack of SLA guarantees.

Production with Inference Endpoints: Secure, Scalable, Pay‑as‑You‑Go

Inference Endpoints provide a managed, production‑grade deployment layer for any Hub model. Users select AWS or Azure regions, choose CPU or GPU instances, and enable auto‑scaling. Pricing starts at $0.06 per hour.

Security Levels

Level Access Typical Use
Public Open internet access, no auth Public demos, open APIs
Protected Requires valid Hugging Face token Controlled public APIs
Private Isolated subnet, reachable only via private cloud link Strict compliance, internal services

The endpoint UI displays health, scaling, and cost metrics, simplifying cost‑performance tuning.

"Inference Endpoints let you deploy any hub model on secure and scalable infrastructure, hosted in your AWS or Azure region of choice. Additional settings include CPU and GPU hosting, built‑in auto‑scaling, and more. This makes finding the appropriate cost/performance ratio easy."

Spaces: Turn Models into Interactive Apps

Spaces combine model inference with a front‑end UI framework (e.g., Gradio) to create shareable demos or lightweight production services. Users can upgrade hardware to advanced Intel CPUs or NVIDIA GPUs directly from the UI.

Key points:

  • Provides a full web UI for model interaction.
  • Supports hardware upgrades for performance.
  • Ideal for showcasing models or building simple internal tools.

Getting Started: From Exploration to Deployment

  1. Log in to the Hugging Face Hub and browse the model catalog.
  2. Use the Inference Widget on a model page to test predictions instantly.
  3. Click Deploy to generate code for the free Inference API.
  4. For production, choose Inference Endpoints or Spaces from the same deployment flow.
  5. Provide feedback on the Hugging Face forum.

Context and Impact

These offerings lower the barrier between research and production, allowing developers to iterate rapidly on state‑of‑the‑art models—including Transformers, Diffusers, and other architectures—while controlling costs via Intel‑sponsored CPU inference. The tiered approach (free widget/API → paid endpoints/Spaces) aligns with typical ML project lifecycles, from proof‑of‑concept to enterprise deployment.

For more details, see the official Hugging Face blog post and the linked documentation for each service.

Sources