Optimizing LLMs in Production: Precision, Attention, and Architecture
Hugging Face outlines key techniques for efficient LLM deployment, focusing on lower precision quantization, Flash Attention for memory efficiency, and architectural optimizations like RoPE, ALiBi, MQA, and GQA.
Fine-tuning Llama 2 70B using PyTorch FSDP
Hugging Face demonstrates how to fine-tune Llama 2 70B using PyTorch Fully Sharded Data Parallelism (FSDP) and Accelerate to overcome CPU RAM bottlenecks and optimize VRAM usage.
Introducing Würstchen: Fast Diffusion for Image Generation
Würstchen is a fast and efficient text-to-image diffusion model that achieves 42x spatial compression to significantly reduce training and inference costs.
Hugging Face Transformers Quantization Overview
Hugging Face provides native support for bitsandbytes and auto-gptq quantization schemes to enable large model inference on smaller devices and efficient adapter fine-tuning.
SafeCoder vs. Closed-source Code Assistants
Hugging Face introduces SafeCoder, an enterprise-grade code assistant based on the open-source StarCoder models that prioritizes transparency, customization, and data privacy over closed-source alternatives.
Efficient Controllable Generation for SDXL with T2I-Adapters
Hugging Face and TencentARC introduce T2I-Adapter-SDXL, a lightweight plug-and-play model that enables precise control over Stable Diffusion XL (SDXL) generation using external signals like sketches and depth maps with significantly lower computational overhead than ControlNet.
Falcon 180B Release Notes
TII has released Falcon 180B, the largest openly available language model with 180 billion parameters, trained on 3.5 trillion tokens to rival proprietary models like PaLM-2.
Fetch Case Study: Reducing ML Processing Latency by 50% with Amazon SageMaker and Hugging Face
Fetch reduced ML processing latency for receipt scans by 50% and increased document-understanding model accuracy by 200% by migrating its ML pipeline to Amazon SageMaker and Hugging Face.
AudioLDM 2 Optimization Guide: Reducing Inference Time with Hugging Face Diffusers
Hugging Face demonstrates how to reduce AudioLDM 2 inference time by over 10x, bringing generation of a 10-second audio sample down to under 1 second using code and model optimizations.
Hugging Face Hub Git Authentication Changes
Hugging Face deprecated password-based Git authentication on October 1, 2023, requiring users to switch to personal access tokens or SSH keys for improved security.
Code Llama Release Notes
Code Llama is a family of open-access models based on Llama 2, specialized for code tasks with support for infilling and long-context windows up to 100,000 tokens.
Hugging Face AutoGPTQ and Transformers Integration
Hugging Face has integrated the AutoGPTQ library into Transformers, enabling the quantization of LLMs to 8, 4, 3, or 2-bit precision to reduce memory requirements with negligible accuracy loss at 4-bit.
IDEFICS: An Open Reproduction of State-of-the-art Visual Language Model
Hugging Face has released IDEFICS, an open-access visual language model based on the Flamingo architecture that supports interleaved image and text inputs in 9B and 80B parameter sizes.
Hugging Face SafeCoder Announcement
Hugging Face has introduced SafeCoder, a self-hosted, enterprise-grade code assistant solution that allows companies to build and deploy proprietary Code LLMs within their own secure infrastructure.
Hugging Face Hub on AWS Marketplace
Hugging Face has integrated its Hub with the AWS Marketplace, allowing organizations to pay for managed services like Inference Endpoints and AutoTrain directly through their AWS accounts.
Deploying DeepFloyd IF with BentoML
Hugging Face demonstrates how to use BentoML to deploy the DeepFloyd IF text-to-image model, enabling independent scaling of its multi-stage pixel diffusion architecture across multiple GPUs.
Optimizing Bark using 🤗 Transformers
Hugging Face demonstrates how to reduce the memory footprint and improve inference speed of the Bark text-to-speech model using Better Transformer, half-precision (fp16), and CPU offload.
Swift Transformers Release: Run On-Device LLMs on Apple Devices
Hugging Face released the alpha swift-transformers Swift package, swift-chat demo app, updated exporters and transformers-to-coreml tools, and Core ML versions of Llama 2 7B and Falcon 7B, enabling developers to run LLMs on-device in Apple apps.
Fine-tuning Llama 2 with Direct Preference Optimization (DPO) via TRL
Hugging Face has integrated Direct Preference Optimization (DPO) into the TRL library, enabling the alignment of Llama 2 and other LLMs using a simplified binary cross-entropy loss instead of complex RLHF pipelines.
Deploying MusicGen via Hugging Face Inference Endpoints
Hugging Face demonstrates how to deploy MusicGen, a text-to-music generation model, as an API using Inference Endpoints and custom handlers for models not supported by standard pipelines.
Towards Encrypted Large Language Models with FHE
Hugging Face and Zama demonstrate how Fully Homomorphic Encryption (FHE) can enable Large Language Models to process encrypted user data while protecting the model owner's intellectual property.
Huggy Lingo: Improving Hugging Face Hub Language Metadata with ML
Hugging Face is using the fastText language identification model and Librarian-Bots to automatically detect and suggest language metadata for datasets that currently lack it.
Practical 3D Asset Generation Guide
Hugging Face provides a step-by-step workflow for creating low-fidelity, PS1-style 3D assets by combining OpenAI's Shap-E model with Blender and Dream Textures.
Segmind SD-Small and SD-Tiny Knowledge Distillation Release
Segmind open-sourced the code and pretrained weights for SD‑Small and SD‑Tiny, two diffusion models that are 35% and 55% smaller than Stable Diffusion while delivering up to 2× faster inference.
Stable Diffusion XL on Mac with Advanced Core ML Quantization
Hugging Face and Apple have ported Stable Diffusion XL to Core ML, introducing mixed-bit palettization to reduce the UNet size by 71% while maintaining image quality on Apple Silicon Macs.
Agents.js Release: Give Tools to LLMs using JavaScript
Hugging Face has released Agents.js, a JavaScript library that enables LLMs to access and execute tools in the browser or on a server.
Hugging Face and Partners Outline Open ML Considerations for the EU AI Act
Hugging Face, in coalition with Creative Commons, Eleuther AI, GitHub, LAION, and Open Future, has proposed five key recommendations to ensure the EU AI Act supports open-source machine learning development.
Hugging Face Open Source AI Game Jam Results
Hugging Face hosted its first Open Source AI Game Jam, resulting in 88 game submissions that integrated open-source AI tools for assets, coding, and gameplay mechanics.
Hugging Face Diffusers First Anniversary Overview
Hugging Face celebrates the first anniversary of the Diffusers library, highlighting its evolution into a modular toolbox for text-to-image, video, 3D, and audio generation.
Llama 2 Release and Hugging Face Integration
Meta has released Llama 2, a family of open-access large language models available for commercial use, with full integration into the Hugging Face ecosystem.
Hugging Face AI WebTV: Building an Automated Video and Music Synthesis Stream
Hugging Face developed AI WebTV, an experimental demo that uses Zeroscope and MusicGen to automatically synthesize video and music streams from LLM-generated prompts.
Open-Source Text Generation & LLM Ecosystem at Hugging Face
Hugging Face’s 2023 blog post surveys the open‑source text generation and LLM ecosystem, covering model backgrounds, licensing, serving tools, and parameter‑efficient fine‑tuning.
Fine-tuning Stable Diffusion on Intel Sapphire Rapids CPUs using Hugging Face Diffusers
Hugging Face shows how to fine-tune a Stable Diffusion model on Intel Sapphire Rapids CPUs using textual inversion with just five example images, demonstrating CPU‑based fine‑tuning via Intel Extension for PyTorch and oneCCL.
Making ML-powered web games with Transformers.js
Hugging Face demonstrates how to create Doodle Dash, a real-time sketch detection game that runs entirely in the browser using Transformers.js and a finetuned MobileViT model.
Deploying LLMs with Hugging Face Inference Endpoints
Hugging Face Inference Endpoints is a managed SaaS solution that allows developers to deploy open-source LLMs as production-ready APIs with features like autoscaling, scale-to-zero, and enterprise-grade security.
Hugging Face: Making a Web App Generator with Open ML Models
Hugging Face demonstrates how to build a text-to-webapp generator using NodeJS, the WizardCoder-15B model, and the Hugging Face Inference Endpoints API.
Leveraging Hugging Face for Complex Generative AI Use Cases: Writer Case Study
Hugging Face highlights its partnership with Writer, demonstrating how the company evolved from a user to a customer and open-source contributor while utilizing the Hugging Face Expert Acceleration Program for scalable LLM production.
Accelerating Vision-Language Models: BridgeTower on Habana Gaudi2
Hugging Face demonstrates that fine‑tuning the BridgeTower vision‑language model on Habana Gaudi2 with Optimum Habana yields up to 2.5× speed‑over Nvidia A100 and 1.4× over H100 by using more dataloader workers and hardware‑accelerated image decoding.
Hugging Face Ethics and Society Newsletter #4: Bias in Text-to-Image Models
Hugging Face outlines the complex sources of bias in text-to-image models and proposes a multi-pronged approach using exploration tools, red-teaming, and documentation to detect and mitigate these biases.
Hugging Face Open LLM Leaderboard MMLU Evaluation Analysis
Hugging Face explains how different implementation details in the MMLU benchmark lead to significantly different performance scores and rankings for the same LLMs.
Panel Integration with Hugging Face Spaces
Hugging Face has integrated a Panel template into Hugging Face Spaces, allowing users to build and deploy Python-based dashboards and complex applications effortlessly.
Hugging Face Response to the U.S. NTIA Request for Comment on AI Accountability
On June 12, 2023, Hugging Face submitted a response to the U.S. NTIA’s request for comment on AI accountability, recommending that accountability mechanisms cover the full ML lifecycle, combine internal documentation with external transparency, and involve a broad range of stakeholders.
Fine-Tune MMS Adapter Models for low-resource ASR
Hugging Face shows how to fine-tune Massive Multilingual Speech (MMS) adapter layers on low-resource ASR data, achieving low word error rates in minutes while keeping the large pretrained model frozen.
Autoformer – Transformers are effective for time series forecasting
Hugging Face announced that the Autoformer model is now available in the 🤗 Transformers library, showing that Transformer‑based models outperform simple linear baselines like DLinear on time‑series forecasting benchmarks.
Hugging Face Content Guidelines and Policy Update
Hugging Face has updated its Content Policy to address the unique challenges of moderating machine learning artifacts and to establish consent as a core value for protecting user rights and privacy.
Deploying Livebook Notebooks as Apps to Hugging Face Spaces
Hugging Face now supports the deployment of Livebook notebooks as interactive applications to Hugging Face Spaces, enabling Elixir-based machine learning apps to be shared easily.
Faster Stable Diffusion with Core ML on iPhone, iPad, and Mac
Hugging Face and Apple have introduced 6-bit palettization and updated attention layer optimizations in Core ML to significantly reduce memory usage and increase inference speed for Stable Diffusion on Apple devices.
Hugging Face and AMD Partnership for Model Acceleration
Hugging Face and AMD have partnered to optimize state-of-the-art transformer models for AMD CPUs, GPUs, and AI accelerators to improve cost-performance standards for training and inference.
The Hugging Face Hub for Galleries, Libraries, Archives and Museums
Hugging Face published a guide showing how galleries, libraries, archives, and museums can use its Hub to share models, datasets, and demos, lowering barriers to machine learning in the GLAM sector.
Can Foundation Models Label Data Like Humans? Hugging Face Analysis
Hugging Face investigated whether GPT-4 can reliably replace human labelers for LLM preference evaluation, finding significant positional and verbosity biases that limit its accuracy as a proxy for human judgment.