Kage: Creating Script-Free Offline Website Mirrors

Kage is a technical tool designed to solve the problem of "broken" offline saves. While traditional "Save As" functions often leave behind dependencies on external JavaScript or analytics servers that eventually fail, Kage creates a permanent, script-free snapshot of a website by rendering it in a real browser and stripping all executable code.

How Kage Works: From Live Site to Static Mirror

Kage converts dynamic websites into static, offline-ready folders through a multi-stage pipeline:

  1. Rendering: Kage uses a pool of headless Chrome tabs to render the target pages. This ensures that content generated by JavaScript is captured as a human would see it.
  2. Snapshotting: Once the page settles, Kage takes a snapshot of the final DOM.
  3. Sanitization: Every script, event handler, and URL is stripped from the DOM to ensure no code runs offline.
  4. Localization: CSS, images, and fonts are downloaded and rewritten to local paths.

This process results in a directory of .html files and a reserved _kage folder containing assets and crawl state. Because the mapping from URL to local path is deterministic, links are rewritten before assets even finish downloading.

Core Functionality and Commands

Kage provides a CLI for cloning, serving, and packaging website mirrors:

Cloning and Mirroring

kage clone <url> performs a breadth-first crawl of a site. It respects robots.txt, seeds from sitemap.xml, and is idempotent—meaning the same page reached via different protocols (HTTP vs HTTPS) is only fetched once.

Key configuration flags include:

  • --max-pages and --max-depth: Limit the scope of the crawl.
  • --scope-prefix: Restrict the crawl to specific paths (e.g., /doc).
  • --scroll: Automatically scrolls pages to trigger lazy-loaded images.
  • --refresh: Re-renders existing mirrors to update content.

Serving and Previewing

Because absolute paths and assets are localized, kage serve [dir] runs a local HTTP server to ensure links and assets resolve correctly, mimicking the original host's behavior.

Packaging for Distribution

To avoid the inefficiency of moving thousands of small files, kage pack collapses a mirror into one of two formats:

  • ZIM Archive: An open, compressed, read-only format used by the Kiwix ecosystem. ZIM files are platform-independent and can be opened in any ZIM reader, ensuring long-term accessibility without vendor lock-in.
  • Self-Contained Binary: By using the --format binary flag, Kage glues the archive onto a copy of the Kage executable. The resulting binary serves the site offline when run, requiring no external dependencies or ZIM readers on the target machine.

Advanced Deployment Options

Native Window Support

By building Kage with the webview tag (CGO_ENABLED=1 go build -tags webview ./cmd/kage), the tool can open packed sites in a native OS window (using WKWebView on macOS, WebView2 on Windows, and WebKitGTK on Linux) rather than a browser tab. This transforms a mirrored website into a standalone application experience.

Containerized Execution

For users who do not want to install Chrome/Chromium locally, Kage provides a Docker image (ghcr.io/tamnd/kage) that bundles Chromium, allowing for isolated cloning via: docker run --rm -v "$PWD/out:/out" ghcr.io/tamnd/kage clone paulgraham.com

Community Insights and Technical Trade-offs

Discussion among users highlights several critical considerations for those using Kage for archival purposes:

Comparison to Existing Tools

Users noted several alternatives depending on the specific use case:

  • SingleFile: Mentioned as a more robust option for packing a single page into one HTML file using base64 encoding for assets.
  • HTTrack: A long-standing tool for downloading wikis and sites, though Kage's use of headless Chrome gives it an advantage with JavaScript-heavy sites.
  • Pandoc: Suggested for those wanting to convert web pages to EPUB for eReaders.

Potential Limitations

  • Interactivity: Because Kage strips all JavaScript, any UI elements relying on JS (dropdowns, search bars, interactive modals) will cease to function in the offline mirror.
  • Security: Some users pointed out that running the browser with --no-sandbox (as seen in some configurations) may pose security risks.
  • Fidelity: One user reported broken images and Unicode symbols when viewing a Kage-generated ZIM file in Kiwix, suggesting that while the format is standard, rendering fidelity can vary by reader.
  • Distribution: The use of self-contained binaries for data storage was criticized by some as a security risk, as users are generally hesitant to run unknown binaries found online.

Suggested Enhancements

Community members suggested adding features to remove cookie banners and annoying pop-ups during the rendering phase to improve the cleanliness of the archived snapshots.

Sources