aavetis/PRarena

This repo tracks the opened and merged PRs by the top SWE coding agents by OpenAI, GitHub, and others. Updates regularly.

What it solves

PRarena provides a way to compare the performance of different AI coding agents (such as Copilot, Codex, Cursor, Devin, and Jules) by analyzing their pull request (PR) statistics on GitHub. It solves the problem of understanding which AI agents are more effective at producing mergeable code across different developer workflows.

How it works

The project uses GitHub search queries to track the number of All PRs, Ready PRs, and Merged PRs created by specific AI agents. By focusing on "Ready PRs" (non-drafts) to ensure a fair comparison, it calculates a success rate based on the ratio of merged PRs to ready PRs.

Who it’s for

Developers and researchers interested in benchmarking AI coding assistants and comparing their real-world merge rates and volume of contributions.

Highlights

  • own interactive dashboard at prarena.ai
  • Tracks multiple AI agents including Copilot, Codex, Cursor, Devin, Codegen, and Jules.
  • Distinguishes between draft and ready PRs to account for different agent workflows.
  • Uses real-world GitHub data to measure success rates.