Hugging Face and FriendliAI Partnership for Model Deployment

Hugging Face and FriendliAI have partnered to integrate FriendliAI's inference infrastructure directly into the Hugging Face Hub. This collaboration allows developers to deploy generative AI models using a "Deploy this model" button on model cards, reducing the friction between model discovery and production serving.

High-Performance Inference Infrastructure

FriendliAI provides GPU-based generative AI inference optimized for speed and cost-efficiency. According to Artificial Analysis, FriendliAI is ranked as the fastest GPU-based generative AI inference provider. This performance is driven by several core technical optimizations:

  • Continuous Batching: Optimizes how requests are processed to increase throughput.
  • Native Quantization: Reduces model size and memory requirements without significant loss in performance.
  • Best-in-class Autoscaling: Dynamically adjusts resources to match demand, reducing latency and operational costs.

One-Click Deployment Workflow

The integration enables a seamless transition from the Hugging Face Hub to FriendliAI's deployment environment. Users can now select Friendli Endpoints from the deployment options on a model card, which redirects them to the FriendliAI model deployment page.

From this interface, users can deploy open-source or custom generative AI models using a Friendli Suite account. The workflow simplifies the process of moving a model from the Hub to a managed inference service without requiring manual infrastructure configuration.

Deployment Options: Dedicated vs. Serverless Endpoints

FriendliAI offers two primary deployment paths for different developer needs:

Friendli Dedicated Endpoints

Dedicated Endpoints provide a managed service for deploying models on NVIDIA H100 GPUs. This option is designed for users who require peak performance and full control over their infrastructure. FriendliAI's optimized inference engine allows developers to reduce the total number of GPUs required to maintain high performance, thereby lowering the operational costs associated with H100 clusters.

Friendli Serverless Endpoints

Serverless Endpoints provide a low-cost, high-performance API for inferencing open-source models that have been optimized by FriendliAI. This is a streamlined solution for developers who want to utilize powerful open-source models without managing any underlying hardware or dedicated instances.

Impact on AI Development

This partnership removes the complexities of infrastructure management, allowing developers and researchers to focus on model innovation rather than the logistics of GPU orchestration. By providing direct access to optimized NVIDIA H100 infrastructure and serverless options, the collaboration increases the accessibility of high-performance open-source AI for developers globally.

Sources