meridianlabs-ai/inspect_petri

What it solves

It automates the process of auditing language models to find alignment issues, reward hacking, and other problematic behaviors. It allows researchers to test specific alignment hypotheses by simulating interactions and monitoring model responses.

How it works

Petri uses a system of agents to conduct audits. It takes seed instructions to generate realistic audit scenarios, then orchestrates multi-turn conversations between an auditor model and a target model. To further test behaviors, it can simulate tools and rollbacks. Finally, a judge model uses a consistent rubric to score the resulting transcripts.

Who it’s for

AI safety researchers and developers who need to end-to-end testing for model alignment and behavioral auditing.

Highlights

• Automated generation of audit scenarios from seed instructions.
• Multi-turn orchestration between auditor and target models.
• Support for tool simulation and rollbacks to test model reactions.
• Rubric-based scoring of transcripts via a judge model.