MaxHu-xuan/chat-archive-guard

What is ChatArchiveGuard?

ChatArchiveGuard is a small, pure‑Python command‑line utility that audits a local directory (or a single file) containing exported chat logs. It scans the files without modifying them or sending any data anywhere and reports:

1. Potential secrets or personal‑data patterns – e.g., API keys, bearer tokens, email addresses, phone numbers, credit‑card‑like numbers.
2. Format problems – malformed UTF‑8, broken JSON/JSONL files.
3. SQLite health – runs SQLite’s PRAGMA quick_check(1) on a private copy of the database (including WAL/SHM side‑cars) and looks for readable text in normal and FTS tables.
4. Coverage gaps – tells you whether every file that could be inspected was actually scanned, or if the scan was truncated because of permission issues, resource limits, or unsupported file types.

The tool is meant as a pre‑delivery gate: before you hand a chat export to analysts, backup pipelines, or third‑party services, you can run a quick local audit to catch accidental credential leaks or corrupted files.

---

Key Features (as described in the README)

Feature	How it works	What you see in the output
Secret & PII detection	Regex‑based patterns for private keys, provider tokens, JWTs, email addresses, phone‑like numbers, IPs, ID numbers, Luhn‑validated card numbers, etc.	Category counts such as secret.provider_key or pii.email (no actual values are shown).
Format validation	Checks UTF‑8 validity and parses JSON/JSONL files.	Categories like format.invalid_json or format.invalid_jsonl.
SQLite integrity	Copies the main DB plus any WAL/SHM files to a private temporary directory, opens the copy, runs PRAGMA quick_check(1), and scans text columns (including FTS virtual tables).	Categories under sqlite.* (e.g., sqlite.quick_check_failed).
Directory coverage reporting	Tracks files_seen (all files encountered) vs. files_scanned (files that entered a content check) and flags complete/truncated when limits or errors prevent full coverage.	Boolean flags and numeric counters in the JSON/text report.
Summary‑only mode	--summary-only removes file paths and per‑finding details, leaving only aggregate counts – useful for CI pipelines where you don’t want to expose filenames or data.	Same exit code, but details_omitted/findings_omitted set to true.
Hard resource limits	Caps on file size, number of files, SQLite rows, value size, and total findings to keep scans predictable.	When a limit is hit a special finding is added and truncated becomes true.
Cross‑platform safety	Uses only the Python standard library; on platforms that cannot safely open SQLite without following symlinks (Windows), the tool aborts that file and marks it unsafe rather than risking data leakage.	Category sqlite.sidecar_unsafe on Windows.

---

Who might use it?

• Data‑engineers or ML ops teams that receive chat export dumps (e.g., from Slack, Teams, Discord) and need to ensure no secrets are inadvertently shipped to downstream pipelines.
• Security auditors performing an offline privacy check before archiving or sharing logs.
• CI/CD pipelines that automatically reject a build if a chat export contains any flagged secret or malformed file.
• Individual developers who want a quick, offline sanity‑check before committing a chat archive to version control.

---

How to get started

1. Install (Python 3.11+ required):

   python3 -m venv .venv && . .venv/bin/activate
   python -m pip install chat-archive-guard
   

2. Run a scan on a directory:

   chat-archive-guard /path/to/export          # human‑readable text report
   chat-archive-guard /path/to/export --json   # JSON report
   chat-archive-guard /path/to/export --json --summary-only   # CI‑friendly aggregate only
   

3. Interpret the exit code:
   • 0 – no findings and the scan covered everything.
   • 1 – at least one finding or incomplete coverage.
   • 2 – invalid path/arguments.

---

Limitations & What it doesn’t do

• It does not verify that the chat messages are complete, authentic, or untampered.
• It does not read compressed archives, encrypted files, images, audio, or proprietary binary formats.
• It does not act as a full‑blown DLP or forensic tool; findings are pattern‑based and may include false positives/negatives.
• It does not upload data; the only network activity is optional package installation.

---

Where to find more information

• Source code & issue tracker: https://github.com/MaxHu-xuan/chat-archive-guard
• License: Apache 2.0 (see LICENSE)
• Detailed threat model: THREAT_MODEL.md
• Release notes & changelog: RELEASE_NOTES.md, CHANGELOG.md

---

TL;DR

ChatArchiveGuard is a lightweight, offline Python tool that safely scans chat export files for secrets, personal data, malformed JSON/UTF‑8, and SQLite integrity issues, providing clear aggregate reports and CI‑compatible exit codes. It’s useful for anyone who needs a quick privacy‑first gate before sharing or processing chat logs.