chiefautism/privacy-parser

What it solves

Privacy Parser identifies and extracts personally identifiable information (PII) from text, providing structured data about what was found, where it is located, and its category. While traditional tools simply redact sensitive information, this project allows you to inventory, audit, or label data for compliance (like GDPR/CCPA) or dataset creation.

How it works

It leverages the 1.5B parameter model from OpenAI's Privacy Filter, which produces BIOES logits. These are processed via Viterbi decoding and span merging to identify entities. A regex-based "backstop" is then applied to catch specific patterns (like secrets or account numbers) that the model might miss, ensuring higher accuracy.

Who it’s for

It is designed for developers and data engineers who need to audit logs, secure data exports, or perform pre-commit checks to prevent sensitive information from leaking into codebases or datasets.

Highlights

• Offers three backends: a fast regex-only parser, a raw model-based parser, and a hybrid approach for optimal accuracy.
• Supports eight PII categories including personal names, emails, phone numbers, addresses, account numbers, and secrets.
• Provides structured output (JSON) containing the label, start/end character indices, and the extracted text.
• Includes a CLI tool for quick processing of text files or piped input.