Hugging Face Open LLM Leaderboard Update: Integrating Math-Verify for Improved Math Evaluation
Hugging Face has re-evaluated 3,751 models on the Open LLM Leaderboard using Math-Verify to fix parsing errors and format strictness, resulting in a significant reshuffling of the MATH-Hard rankings.
Hugging Face Integrates Fireworks.ai as an Inference Provider
Hugging Face has added Fireworks.ai as a supported Inference Provider on the Hub, enabling serverless inference for models like DeepSeek-R1 and Llama-3.2-90B-Vision-Instruct across the HF ecosystem.
Hugging Face: Optimizing Cost and Latency for 1 Billion Classifications
Hugging Face provides a framework and benchmarks for reducing the cost of large-scale encoder model inference, demonstrating that NVIDIA L4 GPUs and optimized batch sizes can process 1 billion text classifications for as little as $253.82.
Hugging Face Video Dataset Scripts
Hugging Face has introduced a set of open video dataset scripts designed to simplify the creation of high-quality, filtered datasets for fine-tuning video generation models.
Hugging Face Xet-backed Repositories: Accelerating Hub Transfers with Block-Level Aggregation
Hugging Face is introducing a chunk-based deduplication system using xet-core and hf_xet to accelerate uploads and downloads by 2-3x through block-level aggregation.
Open R1 Update #2: OpenR1-Math-220k Dataset and Reasoning Insights
Hugging Face introduces OpenR1-Math-220k, a large-scale math reasoning dataset designed to reconstruct DeepSeek R1's distillation pipeline, while sharing community insights on GRPO and Chain-of-Thought length control.
The Open Arabic LLM Leaderboard 2
Hugging Face and partners have released the Open Arabic LLM Leaderboard 2 (OALL v2), replacing saturated and machine-translated benchmarks with native Arabic datasets and a new RAG evaluation framework to more accurately measure Arabic language capabilities.
DABStep: Data Agent Benchmark for Multi-step Reasoning
Hugging Face and Adyen have introduced DABStep, a benchmark of over 450 real-world data analysis tasks that reveals current state-of-the-art AI agents achieve only 16% accuracy on complex reasoning tasks.
pi0 and pi0-FAST: Vision-Language-Action Models for General Robot Control
Hugging Face has integrated pi0 and pi0-FAST, generalist Vision-Language-Action (VLA) models developed by Physical Intelligence, into the LeRobot repository to enable versatile robot control across diverse embodiments.
Hugging Face Open-source DeepResearch
Hugging Face has open-sourced a framework to reproduce OpenAI's Deep Research capabilities, achieving a 55.15% score on the GAIA benchmark using a code-native agentic approach.
Open-R1 Update #1: Replicating DeepSeek-R1 Training and Synthetic Data
Hugging Face provides a first progress update on the Open-R1 project, detailing the reproduction of DeepSeek-R1 evaluation scores, the integration of GRPO into TRL, and strategies for scaling synthetic reasoning data generation.
Mini-R1: Reproducing DeepSeek-R1 Reasoning via GRPO and the Countdown Game
Hugging Face demonstrates how to reproduce the DeepSeek-R1 "aha moment" of self-correction and reasoning by training a Qwen2.5-3B model using Group Relative Policy Optimization (GRPO) on the Countdown Game.
Hugging Face AI Tools for Art Newsletter Issue 1
Hugging Face has launched a monthly newsletter detailing the state of open-source creative AI, highlighting 2024's shift to Diffusion Transformers and the emergence of high-quality open video and audio models.
How to Deploy and Fine‑Tune DeepSeek R1 Models on AWS
Hugging Face shows how to deploy and fine‑tune DeepSeek R1 and its distilled variants on AWS using Hugging Face Inference Endpoints, Amazon Bedrock Marketplace, Amazon SageMaker AI (GPU and Neuron instances), and EC2 Neuron with the Hugging Face Neuron Deep Learning AMI.
Open-R1: A Fully Open Reproduction of DeepSeek-R1
Hugging Face has launched the Open-R1 project to systematically reconstruct the data and training pipeline of DeepSeek-R1, aiming to provide the open-source community with the missing datasets and code for reasoning models.
Hugging Face Inference Providers Integration
Hugging Face has integrated four serverless inference providers—fal, Replicate, SambaNova, and Together AI—directly into the Hub to provide unified, model-centric serverless inference.
State of Open Video Generation Models in Diffusers
Hugging Face provides a comprehensive overview of open video generation models and introduces a suite of Diffusers optimizations that can reduce VRAM requirements for models like HunyuanVideo from 60GB to approximately 6.5GB.
smolagents Vision Support Update
Hugging Face has added native vision support to smolagents, enabling the use of Vision Language Models (VLMs) in agentic pipelines for tasks like autonomous web browsing.
NVIDIA KVPress: Toolkit for KV Cache Compression in Long-Context LLMs
NVIDIA has released KVPress, a Python toolkit that implements state-of-the-art KV cache compression techniques to reduce the memory footprint and increase decoding speed of long-context Large Language Models.
SmolVLM 256M and 500M Release Notes
Hugging Face introduces SmolVLM-256M and SmolVLM-500M, delivering highly efficient Vision Language Models that maintain strong multimodal performance in a significantly reduced parameter footprint.
Hugging Face and FriendliAI Partnership for Model Deployment
Hugging Face has integrated FriendliAI's inference infrastructure into the Hugging Face Hub, enabling one-click deployment of generative AI models to high-performance endpoints.
Hugging Face Organization Blog Articles Feature
Hugging Face now allows organizations subscribed to Enterprise Hub to publish blog articles directly to their organization profiles.
Hugging Face Text Generation Inference (TGI) Multi-Backend Support
Hugging Face has introduced a multi-backend architecture for Text Generation Inference (TGI), allowing users to utilize a unified frontend to deploy LLMs via various execution engines like TensorRT-LLM and vLLM.
Hugging Face Transformers timm Integration
Hugging Face has introduced the TimmWrapper, allowing any model from the PyTorch Image Models (timm) library to be used seamlessly within the transformers ecosystem for inference, quantization, and fine-tuning.
Train 400x faster Static Embedding Models with Sentence Transformers
Hugging Face introduces a method to train static embedding models that run 100x–400x faster on CPU while retaining at least 85% of the quality of models like all‑mpnet‑base‑v2, releasing two models (static‑retrieval‑mrl‑en‑v1 and static‑similarity‑mrl‑multilingual‑v1) with training scripts and evaluation results.
Hugging Face AI Agents Ethics and Framework Analysis
Hugging Face provides a comprehensive framework for understanding AI agents, arguing that risks increase with autonomy and recommending against the development of fully autonomous agents.
Hugging Face releases vdr-2b-multi-v1 multilingual visual document retrieval model
Hugging Face has introduced vdr-2b-multi-v1, a multilingual embedding model for visual document retrieval that enables searching complex documents without OCR by encoding page screenshots into dense vectors.
Hugging Face Open LLM Leaderboard: CO₂ Emissions and Model Performance Analysis
Hugging Face reveals that community fine-tuned models often exhibit higher carbon efficiency than official releases due to increased conciseness and improved instruction following.
Hugging Face smolagents Release
Hugging Face has launched smolagents, a lightweight library that enables LLMs to perform complex tasks by writing actions as executable code rather than JSON, improving composability and generality.
Visualize and understand GPU memory in PyTorch – Hugging Face Blog Summary
Hugging Face’s blog post explains how to visualize GPU memory usage in PyTorch using torch.cuda.memory tools, breaks down memory components during model training, and provides formulas to estimate total memory requirements.
NVIDIA LogitsProcessorZoo: Controlling Language Model Generation with Modular Logits Processors
NVIDIA's LogitsProcessorZoo provides modular logits processors for Hugging Face Transformers that let developers control generation length, enforce phrase inclusion, cite prompt content, and restrict outputs to multiple-choice choices.
Big Bench Audio Release
Artificial Analysis has released Big Bench Audio, a dataset of 1,000 audio questions designed to evaluate the reasoning capabilities of audio language models, revealing a significant performance gap between text and speech reasoning.
ModernBERT Release Notes
Hugging Face, Answer.AI, and LightOn have released ModernBERT, a state-of-the-art encoder-only model family that improves upon BERT's speed, accuracy, and context length to 8,192 tokens.
Bamba-9B: Inference-Efficient Hybrid Mamba2 Model
Bamba-9B, a hybrid Mamba2 model from IBM, Princeton, CMU, and UIUC trained on 2.2T open tokens, achieves 2.5x throughput and 2x latency improvements over Llama 3.1 8B in vLLM and is immediately usable in transformers, vLLM, TRL, and llama.cpp.
Benchmarking Language Model Performance on 5th Gen Xeon at GCP
Hugging Face benchmarked text embedding and generation on Google Cloud's C4 (5th‑gen Xeon) and N2 (3rd‑gen Xeon) instances, finding C4 delivers 10‑24× higher embedding throughput and 2.3‑3.6× higher generation throughput, yielding 7‑19× and 1.7‑2.9× total‑cost‑of‑ownership advantages respectively.
Falcon 3 release notes / what's new
Technology Innovation Institute (TII) has released Falcon 3, a family of decoder-only large language models under 10 billion parameters designed for high efficiency and enhanced science, math, and coding capabilities.
Hugging Face Synthetic Data Generator
Hugging Face has introduced the Synthetic Data Generator, a no-code application that allows users to create custom text classification and chat datasets using natural language prompts.
LeMaterial v1.0: LeMat-Bulk dataset release
LeMaterial v1.0 launches as an open-source initiative releasing the LeMat-Bulk dataset, which unifies 6.7M entries from Materials Project, Alexandria, and OQMD into a standardized format with seven properties to accelerate materials discovery.
Hugging Face Open Preference Dataset for Text-to-Image Generation
The Data is Better Together community has released an Apache 2.0 licensed open preference dataset for text-to-image generation to address the lack of open-source preference data for model alignment.
Hugging Face Models in Amazon Bedrock Marketplace
Hugging Face has integrated 83 open models into the Amazon Bedrock Marketplace, allowing AWS customers to deploy open models on managed infrastructure while maintaining compatibility with Bedrock APIs.
PaliGemma 2 Release Notes
Google has released PaliGemma 2, a new family of vision language models that combine the SigLIP image encoder with the Gemma 2 text decoder across three parameter sizes and multiple input resolutions.
How good are LLMs at fixing their mistakes? A chatbot arena experiment with Keras and TPUs
Hugging Face tested several sub‑10B LLMs on a simple calendar‑API task and found that Gemma 2 9B consistently fixed mistakes with minimal prompting, while smaller and older models struggled or required many corrective turns.
AraGen Benchmark and Leaderboard: Introducing 3C3H Evaluation for Arabic LLMs
Hugging Face introduced AraGen, a dynamic benchmark and leaderboard for Arabic LLMs that uses the 3C3H measure to evaluate correctness, completeness, conciseness, helpfulness, honesty, and harmlessness.
Hugging Face CFM Case Study: Fine-tuning Small Models with LLM Insights
Capital Fund Management (CFM) improved financial Named Entity Recognition (NER) accuracy by up to 6.4% and reduced inference costs by up to 80x by using Llama 3.1 to assist in labeling data for fine-tuning compact models like GLiNER and SpanMarker.
Open Source Developers Guide to the EU AI Act
The Hugging Face guide explains how the EU AI Act applies to open source AI developers, outlining obligations for limited‑risk AI systems and non‑systemic‑risk general purpose AI models and pointing to tools for compliance.
Hugging Face Hub Storage Rearchitecture
Hugging Face is redesigning its upload and download architecture by introducing a content-addressed store (CAS) to enable byte-level deduplication and improve global transfer speeds for massive AI models.
SmolVLM release notes / what's new
Hugging Face introduces SmolVLM, a 2B parameter Vision Language Model (VLM) that is fully open-source and optimized for low memory footprints and high throughput on edge devices.
You could have designed state of the art positional encoding
The Hugging Face blog post walks through an iterative design of positional encoding for transformers, showing how sinusoidal encoding leads to Rotary Positional Encoding (RoPE) and why it matters for modeling token relationships.
Hugging Face and LLM-jp Launch Open Japanese LLM Leaderboard
Hugging Face and LLM-jp have introduced the Open Japanese LLM Leaderboard, a transparent evaluation platform featuring over 20 datasets to benchmark the performance of Japanese large language models.
Hugging Face Introduces Content-Defined Chunking to Improve Storage Efficiency
Hugging Face announced a content-defined chunking storage approach via its Xet team that reduces storage and transfer costs for large model and dataset files by only uploading modified chunks.