Experimenting with the Cross-Origin Storage API in Transformers.js

Experimenting with the Cross-Origin Storage API in Transformers.js

The Cross‑Origin Storage (COS) API lets web apps store and retrieve large files by cryptographic hash instead of by URL, allowing origins to share cached resources without the browser’s origin‑partitioned cache blocking hits. By enabling an experimental flag in Transformers.js, developers can avoid re‑downloading the same model weights and Wasm runtime on every site, saving bandwidth and storage.

The cache isolation problem in Transformers.js

Browsers isolate HTTP caches by origin to prevent timing attacks, so identical resources fetched from different sites are stored separately. In the Transformers.js demo, visiting an ASR demo on https://googlechrome.github.io caches the Whisper model (177 MB) and the ONNX Runtime Wasm file (4,733 kB). Loading the same demo from a different origin such as https://rawcdn.rawgit.net forces the browser to download and cache those resources again, even though the bytes are identical.

How the Cross‑Origin Storage API solves it

COS introduces navigator.crossOriginStorage, which identifies files by a hash (e.g., SHA‑256) rather than by URL. When a site requests a file by its hash, the API returns a FileSystemFileHandle if the hash is already present in the store; otherwise it falls back to the network and writes the file into COS for future use. Because the key is the hash, any origin that has previously stored the same bytes obtains a cache hit, eliminating duplicate downloads.

Integrating COS in Transformers.js

Transformers.js provides an opt‑in backend that uses COS when the flag env.experimental_useCrossOriginStorage is set to true. The library computes the SHA‑256 hash of each Xet‑tracked model file (e.g., the ONNX weight files) from the raw pointer stored on the Hugging Face Hub and passes that hash to navigator.crossOriginStorage.requestFileHandle. If the file exists in COS it is read instantly; if not, it is downloaded and stored for the next caller. Enabling the feature requires only one line before the first pipeline call:

import { env, pipeline } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@4.2.0\); // 👇 Opt in to the experimental Cross‑Origin Storage cache backend.
env.experimental_useCrossOriginStorage = true;
const asr = await pipeline('automatic-speech-recognition', 'Xenova/whisper-tiny.en', { device: 'webgpu' });
const result = await asr('jfk.wav');
console.log(result);

Controlling visibility and integrity

When storing a file via COS, developers can specify the origins option:

  • origins: '*' makes the resource globally available, suitable for shared model weights or the Wasm runtime.
  • A list like origins: ['https://write.example.com', 'https://calculate.example.com'] restricts access to those sites.
  • Omitting origins limits the file to same‑site origins. Visibility can only be upgraded (e.g., from restricted to global) and never downgraded, preventing a malicious site from shrinking the audience of a public resource. The API also verifies the supplied data against the declared hash on write; a mismatch causes an error, giving automatic integrity checking without extra code.

Privacy considerations

Because any site can probe for a file by hash, COS includes availability gating: the browser may conceal the presence of a file that has been seen on only a few origins to prevent fingerprinting. An error from requestFileHandle therefore does not definitively mean the file is absent; it may indicate the browser is withholding confirmation. Apps should treat the error as a cache miss and fall back to the network.

Trying it out

To experiment today, install the Cross‑Origin Storage extension from the Chrome Web Store, which injects a polyfill for navigator.crossOriginStorage. With the extension active, open the ASR demo with COS enabled on https://googlechrome.github.io/samples/transformersjs-automatic-speech-recognition/index3.html, let it load the Whisper model, then open the same demo from a different origin (https://rawcdn.rawgit.net/GoogleChrome/samples/1e4f2b8c10adc394352c6ec8327bb503bac7aba1/transformersjs-automatic-speech-recognition/index3.html). Instead of the 177 MB re‑download observed without COS, the model is served from COS in milliseconds. The extension’s popup shows the shared resource identified by its SHA‑256 hash and the two origins that have it stored.

Call to action

If you are building a Transformers.js app, add env.experimental_useCrossOriginStorage = true before your first pipeline() call, install the COS extension, and verify that duplicate downloads disappear from the Network tab. Every site that opts in makes the experience faster and cheaper for all other sites’ users. The opt‑in is risk‑free: if the COS API is unavailable (no extension), the code falls back to the default Cache API. Feedback on the API can be sent via the Cross‑Origin Storage repository on GitHub, where the Chrome team is also considering a native implementation.

Sources