facebook/ThreatExchange

What it solves

This repository provides tools for identifying and blocking harmful content across the internet. It solves the problem of efficiently detecting duplicate or similar images and videos (content moderation) and sharing security threat intelligence (malware, phishing) between organizations.

How it works

The project consists of several specialized tools:

• Hashing Algorithms: PDQ for images and TMK/vPDQ for videos convert media files into unique digital signatures (hashes). These signatures allow systems to find similar content even if the files are slightly different.
• HMA (Hasher-Matcher-Actioner): A deployment-ready platform for AWS that manages lists of known harmful content and scans new media against those lists.
• python-threatexchange: A library and command-line tool that implements content scanning and facilitates the exchange of threat signals via Meta's API.
• ThreatExchange API: A RESTful system for querying and publishing security threat information like phishing pages and malware.

Who it’s for

Trust and safety teams, security researchers, and organizations that need to automate content moderation or share threat intelligence to protect users.

Highlights

• Multi-modal hashing: Specialized algorithms for both images (PDQ) and videos (TMK, vPDQ).
• Scalable infrastructure: Includes the HMA platform for AWS and a cloud-agnostic Docker-based version (Open Media Match).
• Collaborative intelligence: Integrated with Meta's ThreatExchange API for community-wide sharing of security threats.