Transformers.js v3 release adds WebGPU acceleration, expanded model support, and server‑side JavaScript compatibility

TL;DR

Transformers.js v3 adds WebGPU acceleration (up to 100× faster than WASM), new quantization formats, support for 120 model architectures, and compatibility with Node.js (ESM + CJS), Deno, and Bun, making high‑performance inference possible directly in browsers and server‑side JavaScript runtimes.

Quick installation

You can install the library from NPM:

npm i @huggingface/transformers

Import the pipeline either via ES modules or a CDN:

import { pipeline } from "@huggingface/transformers";
// or
import { pipeline } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.0.0";

Full documentation is available at https://hf.co/docs/transformers.js.

WebGPU support – massive speed boost

WebGPU is a modern web standard that exposes the GPU for general‑purpose compute, superseding WebGL. As of October 2024, about 70 % of browsers support WebGPU, though some may require feature flags (Firefox dom.webgpu.enabled, Safari WebGPU, older Chromium enable-unsafe-webgpu).

Enabling WebGPU in Transformers.js

The library integrates with ONNX Runtime Web; you enable GPU acceleration simply by passing device: "webgpu" when loading a model.

Text‑embedding example

const extractor = await pipeline(
  "feature-extraction",
  "mixedbread-ai/mxbai-embed-xsmall-v1",
  { device: "webgpu" }
);
const embeddings = await extractor(["Hello world!", "Example sentence."], { pooling: "mean", normalize: true });
console.log(embeddings.tolist());

Whisper ASR example

const transcriber = await pipeline(
  "automatic-speech-recognition",
  "onnx-community/whisper-tiny.en",
  { device: "webgpu" }
);
const output = await transcriber("https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav");
console.log(output);

Image classification example

const classifier = await pipeline(
  "image-classification",
  "onnx-community/mobilenetv4_conv_small.e2400_r224_in1k",
  { device: "webgpu" }
);
const result = await classifier("https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/tiger.jpg");
console.log(result);

These demos show that inference that previously required seconds on WASM now runs in real time on the GPU.

New quantization formats (dtype parameter)

Previously the library exposed a binary quantized flag (q8 vs fp32). Version 3 replaces this with a flexible dtype argument that can select from many precisions, including:

  • Full‑precision: "fp32"
  • Half‑precision: "fp16"
  • 8‑bit: "q8", "int8", "uint8"
  • 4‑bit: "q4", "bnb4", "q4f16"

Basic usage – 4‑bit Qwen2.5

const generator = await pipeline(
  "text-generation",
  "onnx-community/Qwen2.5-0.5B-Instruct",
  { dtype: "q4", device: "webgpu" }
);
const msgs = [{ role: "system", content: "You are a helpful assistant." }, { role: "user", content: "Tell me a funny joke." }];
const out = await generator(msgs, { max_new_tokens: 128 });
console.log(out[0].generated_text.at(-1).content);

Per‑module dtypes for encoder‑decoder models

Some models (e.g., Whisper, Florence‑2) are sensitive to quantization. You can now provide a mapping from module name to dtype:

const model = await Florence2ForConditionalGeneration.from_pretrained(
  "onnx-community/Florence-2-base-ft",
  {
    dtype: {
      embed_tokens: "fp16",
      vision_encoder: "fp16",
      encoder_model: "q4",
      decoder_model_merged: "q4",
    },
    device: "webgpu",
  }
);

The full example loads the model, processor, tokenizer, prepares an image, and generates a detailed caption, demonstrating the practical impact of per‑module quantization.

120 supported architectures – broader modality coverage

Transformers.js v3 now supports 120 model families, spanning text, vision, audio, and multimodal tasks. Notable additions include:

  • Phi‑3 and Phi‑3.5 (high‑capability LLMs that run on‑device)
  • Gemma and Gemma 2 (Google’s open models)
  • LLaVA, Moondream, Florence‑2 (vision‑language)
  • MusicGen (audio generation)
  • Depth Pro, RT‑DETR, Sapiens, PyAnnote (specialized vision and audio tasks)

A bubble diagram in the original post visualizes these new families. The full list is available at https://huggingface.co/docs/transformers.js/index#models.

Example projects and templates – showcase of WebGPU

The release ships 25 new example repositories, many demonstrating WebGPU acceleration. Highlights:

  • Phi‑3.5 WebGPU – runs a 3.8 B parameter LLM entirely in the browser.
  • Whisper Turbo WebGPU – real‑time speech‑to‑text using the Whisper model on GPU.

All examples are hosted under https://github.com/huggingface/transformers.js-examples and will be consolidated there.

Over 1,200 pre‑converted models on the Hub

The community has already converted more than 1,200 models to the ONNX format required by Transformers.js. The searchable list is at https://hf.co/models?library=transformers.js.

To convert a custom model, use the provided script:

python -m scripts.convert --quantize --model_id <model_name_or_path>

After uploading, tag the repository with transformers.js for discoverability.

Server‑side JavaScript runtimes support

Transformers.js v3 runs on the three most popular runtimes:

Runtime Compatibility Example repo
Node.js (ESM + CJS) Full support, npm package https://github.com/huggingface/transformers.js-examples/tree/main/node-esm
Deno Secure defaults, experimental WebGPU https://github.com/huggingface/transformers.js-examples/tree/main/deno-embed
Bun High‑performance bundler & runtime https://github.com/huggingface/transformers.js-examples/tree/main/bun

This enables developers to run inference on servers, edge devices, or within serverless functions without Python dependencies.

New home on NPM and GitHub

The package is now published under the official Hugging Face scope as @huggingface/transformers (previously @xenova/transformers).

The source code has moved to https://github.com/huggingface/transformers.js. This centralizes issue tracking, pull‑request contributions, and community engagement under the Hugging Face organization.

Implications

  • Performance: WebGPU brings browser‑based inference into the realm of desktop‑class speed, unlocking real‑time applications such as interactive chat, live transcription, and on‑device vision.
  • Flexibility: The expanded dtype API and per‑module quantization let developers balance memory, latency, and accuracy for a wide range of hardware.
  • Ecosystem growth: Supporting 120 architectures and over 1,200 pre‑converted models lowers the barrier for using state‑of‑the‑art models in JavaScript environments.
  • Portability: Compatibility with Node.js, Deno, and Bun means the same code can run in browsers, serverless back‑ends, or edge runtimes, simplifying deployment pipelines.
  • Community alignment: Moving the package and repository under the Hugging Face organization consolidates branding and encourages contributions from the broader HF ecosystem.

Getting started checklist

  1. Install @huggingface/transformers via npm or CDN.
  2. Choose a model from the Hub (ensure it has a transformers.js tag).
  3. Decide on a device ("cpu", "wasm", or "webgpu").
  4. Optionally select a dtype or per‑module dtype mapping for quantization.
  5. Run inference using the high‑level pipeline API or low‑level model classes.
  6. Deploy the same code to Node.js, Deno, or Bun as needed.

References

This article summarizes the official Hugging Face blog post “Transformers.js v3: WebGPU Support, New Models & Tasks, and More…” published on 2024‑10‑22.

Sources