Anthropic Election Integrity Testing and Mitigation Framework

Anthropic has developed a flexible, iterative process to identify and mitigate elections-related risks in its AI models. By combining in-depth expert testing with large-scale automated evaluations, the company aims to prevent the dissemination of inaccurate election information and stop the use of its models for policy-violating activities such as disinformation campaigns.

A Multi-Layered Approach to Election Safety

Anthropic employs a "Swiss cheese model" of system safety, utilizing layered and overlapping interventions to ensure that no single point of failure compromises election integrity. This process consists of four primary stages: Policy Vulnerability Testing, Automated Evaluations, Implementation of Mitigation Strategies, and Retesting for Efficacy.

Policy Vulnerability Testing (PVT)

Policy Vulnerability Testing is a qualitative, in-depth testing phase conducted with external subject matter experts, including researchers from the Institute for Strategic Dialogue. PVT focuses on two primary risks: users receiving harmful or inaccurate information and users violating the Usage Policy.

The PVT process follows three stages:

Planning: Selecting policy areas, such as election administration, political parity, and potential misuse by bad actors.
Testing: Experts construct both non-adversarial and adversarial prompts to document model outputs and benchmark them against policies.
Reviewing: Collaborative sessions to identify gaps in safety systems and prioritize remediations.

Scalable Automated Evaluations

To provide breadth that manual testing cannot achieve, Anthropic uses automated evaluations. These are developed by using language models to generate hundreds of test questions based on a few-shot prompting approach using expert-written PVT questions.

Automated evaluations allow for:

Scalability: Testing hundreds of prompts across multiple model variations in minutes.
Comprehensiveness: Assessing a wider range of scenarios through large, targeted evaluation sets.
Consistency: Reducing variability by applying a consistent set of questions across different models.

To ensure quality, Anthropic manually reviews samples of these generated questions. In one instance, a review of 64 questions from a 700-question set regarding EU election administration found that 89% of the model-generated questions were relevant extensions of the PVT work.

Risk Mitigation Strategies

Findings from PVT and automated evaluations lead to several specific technical and policy interventions:

System Prompt Updates: Adding context to the system prompt, such as the model's knowledge cutoff date, to help users understand the timeliness of the information provided.
Model Fine-Tuning: Using specific training data to incentivize desired behaviors. For example, Anthropic created a "reward" for the model to refer users to authoritative sources.
Policy Refinement: Updating the Usage Policy to explicitly prohibit generating misinformation, interfering with election processes, or advocating for specific political candidates or parties.
Enforcement Tooling: Utilizing a fine-tuned version of Claude to evaluate prompts and completions in real-time, supplemented by manual audits and the offboarding of users who violate policies.
Direct User Redirection: Implementing UI elements, such as pop-up banners on claude.ai, that redirect US-based users to TurboVote and EU-based users to European Parliament instructions.

Measuring Efficacy through Case Studies

Anthropic uses its testing framework to verify if mitigations actually work by re-running the same protocols after interventions.

Knowledge Cutoff References

To ensure models reference their knowledge cutoff date (August 2023) for time-sensitive election queries, Anthropic updated the system prompt. Comparing Claude 2, a research version of Claude 3 Opus (without the prompt), and the public Claude 3 Opus (with the prompt), Anthropic observed a 47.2% improvement in this priority mitigation.

Referrals to Authoritative Sources

To improve the frequency of referrals to authoritative sources, Anthropic employed model fine-tuning. A comparison between Claude 2 (not fine-tuned for this behavior) and Claude 3 Opus (fine-tuned) showed a 10.4% improvement in how often the model referred users to reliable sources when appropriate.