ttttccxxui/DataInfra-RedactionEverything

DataInfra Series. Redact EVERYTHING with local llms and vlms.

What it solves

RedactionEverything is a local-first document anonymization workbench designed to remove sensitive information from unstructured files (PDFs, Word docs, images, and plain text) without sending data to remote APIs. It addresses the challenge of redacting messy, real-world business documents—particularly bilingual Chinese-English files—that contain not only text but also visual elements like stamps, signatures, and faces.

How it works

The system employs a dual-path processing pipeline:

  1. Text + OCR Path: It uses PP-StructureV3 for layout analysis and OCR, converting images and scanned PDFs into text blocks. A semantic NER model (HaS Text) then identifies sensitive entities based on configurable schemas. It also includes a "red-ink suppression" pass to recover text hidden under red company seals.
  2. Visual Feature Path: It uses the LocateAnything-3B model to ground visual features such as faces, fingerprints, ID cards, and bank cards. A local OpenCV detector supplements this by finding red binding and edge seals.

Results from both paths are merged and deduplicated, allowing users to review, correct, and export the redacted documents locally.

Who it’s for

It is designed for users handling sensitive business, legal, finance, or healthcare documents who require strict privacy boundaries (local/intranet deployment) and need to redact both text and visual identifiers from complex document layouts.

Highlights

  • Local-First Privacy: All inference and file processing happen on a local GPU workstation or intranet.
  • Visual Grounding: Detects non-text sensitive elements like signatures, seals, and faces using a single grounding model.
  • Industry-Specific Presets: Includes pre-configured schemas for Legal, Finance, and Healthcare domains.
  • Comprehensive File Support: Handles TXT, DOCX, scanned PDFs, and various image formats.
  • Human-in-the-Loop: Provides a full workbench for reviewing and correcting recognition results before final export.

Related

  • Project
    egdels/makeacopy

    An offline, privacy-focused Android document scanner that uses on-device ML and OCR to create searchable PDFs without cloud dependencies.

  • Project
    chrisryugj/Docufinder

    Anything is a Windows/macOS desktop app that locally indexes documents (HWP, DOCX, PDF, images, etc.) and lets you search by content, filename, or natural‑language query. Optional AI Q&A and summarisation use Gemini or OpenAI‑compatible endpoints; otherwise the tool works fully offline. Features include fast full‑text search, real‑time folder monitoring, preview with original layout, OCR, markdown export, file‑compare, and a “Lite” build for air‑gapped environments. It is built with React/Tauri on the front‑end and Rust‑based parsers/indexers on the back‑end, and is released under a Business Source License.

  • Project
    pynicolas/FairScan

    An open-source Android document scanner that uses local machine learning for automatic document detection and perspective correction to create private PDFs.

  • Project
    opendatalab/OmniDocBench

    A comprehensive benchmark for evaluating document parsing in real-world scenarios, featuring rich annotations for layout detection, OCR, table, and formula recognition.

  • Project
    magicrew/doc7

    A document conversion tool that turns PDFs, Office files, and images into AI-ready Markdown using multimodal vision models, supporting local inference for privacy and cost-efficiency.