lynote-ai/ai-image-detector

AI Image Detector – What It Is

AI Image Detector is an open‑source command‑line / Python library that tells you how likely a given picture was created by an AI image generator. It wraps several research‑grade detectors (the default is UniversalFakeDetect – a CLIP‑based model with a tiny linear head) and also lets you mix in external detectors such as Nonescape and Sentry. The tool follows the “install‑once‑run‑one‑command” philosophy of utilities like yt‑dlp or rembg.

---

Key Features (as described in the README)

Feature	What It Does
Multiple back‑ends	univfd (default), sentry‑convnext‑small, ultra (ensemble of internal + external detectors), nonescape‑mini, hybrid, hf (any Hugging Face image‑classification checkpoint).
CLI	aidetect detect <path> – single image or recursive folder, with optional --csv, --json, and --backend flags.
Python API	create_detector(backend, device) → predict_path(image_path) returns a result object with probability_ai and a label.
Web UI	Install the web extra and run aidetect serve to get a Gradio interface.
FastAPI server	Install the api extra and run aidetect api … for a simple HTTP endpoint (/detect).
Benchmarking utilities	Commands to evaluate a folder, a Hugging Face dataset, or local Tiny‑GenImage shards; they output CSV/JSON with accuracy, F1, ROC‑AUC, per‑image scores, and threshold sweeps.
Threshold calibration	Built‑in support for optimizing a decision threshold for metrics like balanced accuracy or F1 on a calibration split.
Reproducible reports	All commands can dump a deterministic JSON report that includes model metadata, dataset info, and per‑image predictions.
Lightweight install	Pure‑Python package, works on Python 3.10+, optional extras for evaluation, HF back‑ends, API, or UI.

---

How to Install & Run (quick start)

# Clone the repo and set up a virtual environment
python -m venv .venv
source .venv/bin/activate
pip install -e .               # core package
pip install -e '.[web]'        # optional Gradio UI (or .[api] for FastAPI)

Detect a single image with the default model:

aidetect detect image.jpg

Detect a whole folder and write a CSV report:

aidetect detect ./images --csv report.csv

Run the web UI:

aidetect serve   # then open http://localhost:7860

---

What the Output Means

• probability_ai – a float in [0, 1] representing the model’s estimated chance the image is AI‑generated.
• label – by default probability_ai >= 0.5 → ai, otherwise human.
• Scores near 0.45‑0.55 are weak evidence; the README advises checking multiple back‑ends for critical decisions.

---

Benchmarks & Performance (as reported)

• Small 3‑image sanity check shows accuracies ranging from 33 % (UnivFD) to 67 % (most other back‑ends).
• On a 200‑image calibrated Tiny‑GenImage split, UnivFD reaches 76 % accuracy after threshold tuning, while the Hybrid blend gets ~67 %.
• The strongest configuration, ultra (Hybrid‑plus + Sentry), attains ≈ 86 % accuracy / 0.86 balanced accuracy / 0.85 F1 / 0.92 ROC‑AUC on a 400‑image test set.
• Performance varies by generator; e.g., Midjourney vs. real images yields 84 % accuracy, while older models like VQDM are harder (78 %).

---

Limitations (explicitly listed)

• No detector works universally – new generators, heavy recompression, screenshots, crops, up‑scaling, or adversarial edits can break detection.
• The tool only gives a whole‑image likelihood; it does not localize edited regions.
• Benchmarks are local, small‑scale, and may over‑state real‑world reliability if your data distribution differs.

---

When to Use Which Backend

Backend	When to Pick It
univfd	Want the simplest, smallest dependency set; good baseline.
sentry‑convnext‑small	Need a strong single external detector without ensembling.
ultra	Want the highest practical accuracy out‑of‑the‑box (ensemble of internal + external).
nonescape‑mini	Useful as an extra signal; can be combined in ensembles.
hf	You have a custom Hugging Face image‑classification checkpoint you’d like to try.
hybrid / hybrid‑plus	Want to blend UnivFD with a HF classifier (adjustable weight).

---

Citation

If you use the tool in research, cite the original UniversalFakeDetect paper (CVPR 2023) as shown in the README.

---

Bottom Line

AI Image Detector is a practical, well‑documented utility for estimating the synthetic likelihood of images. It bundles several state‑of‑the‑art detectors, offers easy CLI, API, and web interfaces, and provides built‑in benchmarking and threshold‑calibration tools—all without requiring you to train your own model.