OpenAI Privacy Filter Release

OpenAI Privacy Filter Release

OpenAI has released OpenAI Privacy Filter, an open-weight model designed to detect and redact personally identifiable information (PII) in unstructured text. This tool is designed to be run locally, allowing developers to mask PII without data leaving their local environment, which reduces the risk of exposure during training, indexing, logging, and review pipelines.

Model Architecture and Technical Specifications

Privacy Filter is a bidirectional token-classification model with span decoding. It was developed by adapting an autoregressive pretrained checkpoint into a token classifier over a fixed taxonomy of privacy labels. Instead of generating text, the model labels an input sequence in a single forward pass and decodes coherent spans using a constrained Viterbi procedure.

Key technical properties include:

  • Efficiency: All tokens are labeled in a single pass, making it fast for production use.
  • Context Awareness: The model uses a language prior to detect PII spans based on surrounding context, distinguishing between public information and private individual data.
  • Capacity: The model supports a context window of up to 128,000 tokens.
  • Model Size: The model consists of 1.5B total parameters with 50M active parameters.
  • Configurability: Developers can adjust operating points to trade off precision and recall based on their specific workflow.

PII Detection Categories

Privacy Filter predicts spans across eight specific categories of sensitive information:

  • private_person
  • private_address
  • private_email
  • private_phone
  • private_url
  • private_date
  • account_number (covers banking and credit card numbers)
  • secret (covers passwords and API keys)

These labels are decoded using BIOES span tags to ensure clean and coherent masking boundaries.

Training Methodology

OpenAI developed the model in three primary stages:

  1. Taxonomy Definition: Establishing a set of privacy labels for personal identifiers, contact details, and secrets.
  2. Conversion: Replacing the language modeling head of a pretrained model with a token-classification head and applying a supervised classification objective.
  3. Data Mixture: Training on a combination of public and synthetic data. Model-assisted annotation and review were used to improve coverage in public data, while synthetic data increased diversity across formats and contexts.

Performance and Benchmarks

On the PII-Masking-300k benchmark, Privacy Filter achieves an F1 score of 96% (94.04% precision and 98.04% recall). When corrected for annotation issues identified by OpenAI, the F1 score increases to 97.43% (96.79% precision and 98.08% recall).

Additionally, the model is highly adaptable. Fine-tuning on a small amount of domain-specific data can increase the F1 score from 54% to 96% on domain-adaptation benchmarks.

Limitations and Intended Use

Privacy Filter is intended as a component of a broader privacy-by-design system, not as a standalone anonymization tool, compliance certification, or substitute for policy review.

Limitations include:

  • Taxonomy Dependence: Performance depends on the label taxonomy and decision boundaries used during training.
  • Distribution Shift: Accuracy may vary across different languages, scripts, and naming conventions.
  • Potential for Errors: The model may miss uncommon identifiers or over-redact in short sequences with limited context.

OpenAI states that in high-sensitivity domains like medical, legal, and financial workflows, human review and domain-specific fine-tuning remain essential.

Availability

OpenAI Privacy Filter is released under the Apache 2.0 license and is available on Hugging Face and GitHub for experimentation, customization, and commercial deployment.

Sources