google/magika

Magika – AI‑powered file‑type detection

What it is – Magika is a small, fast deep‑learning model that identifies the content type of a file (e.g., Python source, DOCX document, INI config) with about 99 % accuracy. It works on a single CPU in ~5 ms per file, regardless of file size, because it only looks at a limited slice of the file’s bytes.

Why it matters – Traditional file‑type sniffers rely on file extensions or simple magic‑byte heuristics, which can be fooled or miss obscure formats. Magika’s neural model, trained on ~100 M samples covering 200+ binary and text formats, provides a more reliable signal for security pipelines, content‑policy scanners, and any workflow that needs to know “what this file really is”.

How you can use it

• CLI (Rust‑based) – magika <path>… with options for recursion, JSON output, MIME‑type only, custom formatting, etc.
• Python library – from magika import Magika; Magika().identify_path('file') returns a rich result object.
• JavaScript/TypeScript – an experimental npm package powers the web demo and can be used in Node or browsers.
• Go – work‑in‑progress bindings.

Installation shortcuts

• pipx install magika or pip install magika for the Python package (which also bundles the CLI).
• brew install magika on macOS/Linux.
• cargo install --locked magika-cli for a pure Rust install.
• npm install magika for the JS binding.

Key features

• Tiny model – only a few megabytes, easy to ship.
• Speed – ~5 ms inference per file after a one‑time model load.
• High confidence modes – high‑confidence, medium‑confidence, best‑guess let you trade recall vs. precision.
• Threshold system – per‑type confidence thresholds decide whether to return a specific label or fall back to generic “text”/“binary”.
• Batch processing – can handle thousands of files in one command; -r scans directories recursively.
• Cross‑language bindings – Rust CLI, Python API, JS/TS npm package, upcoming Go support.

Real‑world usage – Google uses Magika at scale to route billions of Gmail, Drive, and Safe‑Browsing files to the appropriate security scanners. It is also integrated into VirusTotal and abuse.ch’s malware‑analysis platforms.

Getting started

# Install the CLI via pipx (includes the Rust binary)
pipx install magika

# Identify a single file
magika myscript.py

# JSON output for automation
magika myscript.py --json

# Scan a whole directory recursively
magika -r /path/to/files

Python example

from magika import Magika
m = Magika()
result = m.identify_path('example.docx')
print(result.output.label)   # → docx
print(result.score)          # confidence score (0‑1)

Where to learn more – The project’s website, the open‑source repository, and the accompanying research paper (ICSE 2025) detail the model architecture, training data, and evaluation methodology.

License & support – Apache 2.0. The project is open source but not an official Google product; security issues should be reported to magika-dev@google.com.