Using Transformers.js in a Chrome Extension

Using Transformers.js in a Chrome Extension

Hugging Face has released a demo browser extension powered by Gemma 4 E2B to demonstrate how to run local AI features within a Chrome extension. The implementation leverages Transformers.js under Manifest V3 constraints, utilizing a decoupled architecture where a background service worker manages model orchestration while the UI and content scripts remain thin clients.

Chrome Extension Architecture (Manifest V3)

The core architectural strategy is the separation of concerns across three primary Chrome runtime contexts to ensure UI responsiveness and avoid duplicate model loading.

Runtime Contexts and Entry Points

As defined in the manifest.json, the extension utilizes three entry points:

  • Background Service Worker (background.js): Acts as the control plane for agent lifecycle, model initialization, and tool execution.
  • Side Panel (sidebar.html): Serves as the interaction layer for chat input/output and streaming updates.
  • Content Script (content.js): Functions as the page bridge for DOM extraction and highlighting actions.

Messaging and Orchestration

Because these runtimes are isolated, a typed messaging contract (defined via enums) coordinates actions. The background service worker acts as the single coordinator:

  1. The side panel sends a request (e.g., AGENT_GENERATE_TEXT).
  2. The background worker appends the message to the conversation history, runs inference, and executes tools.
  3. The background worker emits an update (e.g., MESSAGES_UPDATE) back to the side panel for rendering.

Transformers.js Integration Details

Model Roles and Responsibilities

The extension employs two distinct models to balance reasoning and retrieval:

  • Text Generation (LLM): onnx-community/gemma-4-E2B-it-ONNX (q4f16) handles reasoning and tool-calling decisions.
  • Vector Embeddings: onnx-community/all-MiniLM-L6-v2-ONNX (fp32) generates embeddings for semantic similarity searches in history and website content.

Inference and Caching

All inference is executed within the background service worker using pipeline("text-generation", ...) with a DynamicCache class for consistent KV Caching, and pipeline("feature-extraction", ...) for embeddings.

Hosting inference in the background worker ensures that model artifacts are cached under the extension origin (chrome-extension://<extension-id>) rather than per-website origins, providing a shared cache across the entire installation. Developers must account for the Manifest V3 lifecycle, as service workers can be suspended and restarted, requiring model runtime state to be recoverable.

Agent and Tool Execution Loop

Tool-Calling Mechanism

Transformers.js uses model-specific chat templates to format prompts. For Gemma 4, the model emits a special tool-call token block (e.g., <|tool_call>call:getWeather{location:<|">Bern<|">}<tool_call|>) when it decides to invoke a tool. The extension uses a normalization layer (webMcp) and a parser (extractToolCalls) to convert these model outputs into deterministic executions.

Loop Design (Agent.runAgent)

The extension separates internal model transcripts from user-facing chat messages:

  • Internal Transcript: Contains system, user, tool, and assistant turns used for the generator(...) function.
  • UI Transcript: Contains streamed assistant text, tool execution metadata, and performance metrics.

The execution flow follows a loop: user input is added $\rightarrow$ tokens are streamed $\rightarrow$ tool calls are parsed and executed in the background $\rightarrow$ results are fed back into the prompt $\rightarrow$ the loop repeats until no tool calls remain.

Data Boundaries and Persistence

State is distributed based on lifecycle and access patterns to optimize performance and durability:

  • Conversation State: Stored in background memory (Agent.chatMessages) for fast orchestration.
  • Tool Preferences: Persisted in chrome.storage.local across sessions.
  • Semantic History Vectors: Stored in IndexedDB (VectorHistoryDB) for local retrieval.
  • Extracted Page Content: Managed in a background cache (WebsiteContentManager) keyed by the active URL.

Build and Packaging

To meet Manifest V3 requirements, the project uses a multi-entry build via Vite, ensuring one artifact per Chrome entry point. The content script is kept as a self-contained output to prevent runtime chunk-loading issues, with output names aligned exactly with the manifest.json definitions.

Sources