Transformers.js v4 release notes / what's new

Hugging Face has released Transformers.js v4, which introduces a completely rewritten C++ WebGPU runtime to enable hardware-accelerated AI models to run locally across browsers, Node, Bun, and Deno. This update significantly improves performance, expands model architecture support, and modularizes the library for better production scalability.

WebGPU Runtime and Performance Gains

Transformers.js v4 adopts a new WebGPU runtime rewritten in C++ in collaboration with the ONNX Runtime team. This runtime allows the same codebase to function across diverse JavaScript environments, including desktop applications and server-side runtimes like Node, Bun, and Deno.

To maximize performance in resource-constrained environments, Hugging Face re-implemented models operation by operation using specialized ONNX Runtime Contrib Operators. Key optimizations include:

  • Specialized Operators: Implementation of com.microsoft.GroupQueryAttention, com.microsoft.MatMulNBits, and com.microsoft.QMoE for large language models.
  • BERT Speedups: The adoption of the com.microsoft.MultiHeadAttention operator resulted in a ~4x speedup for BERT-based embedding models.

Expanded Model Support and Architectures

The new export strategy and expanded ONNX Runtime operator support enable Transformers.js v4 to support a wider array of models and advanced architectural patterns, including:

  • Supported Architectures: Mamba (state-space models), Multi-head Latent Attention (MLA), and Mixture of Experts (MoE).
  • New Models: Support for GPT-OSS, Chatterbox, GraniteMoeHybrid, LFM2-MoE, HunYuanDenseV1, Apertus, Olmo3, FalconH1, and Youtu-LLM.
  • Large Model Capability: The library now supports models exceeding 8B parameters; for example, GPT-OSS 20B (q4f16) achieved approximately 60 tokens per second on an M4 Pro Max.

Library Infrastructure and Build System

Transformers.js v4 undergoes a significant structural overhaul to improve maintainability and developer experience:

  • Build System Migration: The transition from Webpack to esbuild reduced build times from 2 seconds to 200 milliseconds (a 10x improvement) and decreased bundle sizes by an average of 10%. The transformers.web.js default export is now 53% smaller.
  • Monorepo Transition: The project now uses pnpm workspaces, allowing the shipment of smaller sub-packages that depend on the @huggingface/transformers core.
  • Code Refactoring: The models.js file was split into smaller, focused modules to replace a single 8,000-line file, improving readability and the process of adding new models.
  • Examples Repository: Example projects have been moved to a dedicated examples repository.

New Production-Ready Features

Several new APIs and settings have been added to support robust application deployment:

ModelRegistry API

The ModelRegistry provides explicit visibility into pipeline assets before loading. Key capabilities include:

  • Listing required files via get_pipeline_files.
  • Inspecting file metadata and calculating download sizes via get_file_metadata.
  • Checking cache status with is_pipeline_cached and clearing artifacts with clear_pipeline_cache.
  • Querying available precision types with get_available_dtypes.
  • Enhanced progress_callback now includes a progress_total event for end-to-end loading progress.

Environment and Logging Controls

  • WASM Caching: env.useWasmCache allows caching of WASM runtime files for offline functionality.
  • Custom Fetch: env.fetch enables custom fetch implementations for authenticated model access or custom headers.
  • Logging: ONNX Runtime WebGPU warnings are hidden by default, and developers can now set explicit verbosity levels using env.logLevel (e.g., LogLevel.DEBUG, LogLevel.INFO, LogLevel.WARNING, LogLevel.ERROR, LogLevel.NONE).

Standalone Tokenizers.js Library

Hugging Face has extracted the tokenization logic into a separate, lightweight library: @huggingface/tokenizers. This standalone library is 8.8kB (gzipped), has zero dependencies, is fully type-safe, and works across browsers and server-side runtimes.

Sources