THU-BPM/MarkLLM

What it solves

MarkLLM addresses the challenge of identifying machine-generated text by providing a standardized toolkit for implementing, visualizing, and evaluating watermarking technologies in Large Language Models (LLMs). It helps researchers and developers ensure the authenticity and origin of AI-generated content.

How it works

MarkLLM provides a unified implementation framework that supports a wide array of watermarking algorithms (such as KGW, SynthID-Text, and MorphMark). It integrates with Hugging Face models and allows users to generate watermarked text and subsequently detect those watermarks. The toolkit also includes visualization tools to explain the underlying mechanisms of the algorithms and an evaluation module to test the watermark's detectability, robustness against attacks (like paraphrasing or word deletion), and its impact on the quality of the generated text.

Who it’s for

It is designed for AI researchers and the broader developer community interested in text watermarking, AI safety, and the detection of machine-generated content.

Highlights

• Extensive Algorithm Support: Implements numerous watermarking methods including KGW, Unbiased, and SynthID-Text.
• Comprehensive Evaluation: Includes 12 tools to measure success rates, text quality (PPL, BLEU), and robustness against various text editors and attacks.
• Unified Interface: Uses an AutoWatermark class to simplify the loading and invocation of different algorithms.
• Visualization Tools: Provides custom tools to demystify how watermarking algorithms operate in different scenarios.