ApodexAI/AgentHarness

What it solves

AgentHarness provides a standardized way to evaluate the performance of AI agents on deep-research tasks. It allows developers to reproduce the benchmark results for Apodex-1.0 and test other models using a consistent ReAct (Reasoning and Acting) setup.

How it works

The harness runs agentic pipelines across a suite of public deep-research benchmarks. It utilizes a subprocess-based execution model where each question is handled in its own isolated process to prevent asyncio saturation and allow for easier debugging and independent reruns of failed samples. It integrates with external tools for web search, web fetching, and code sandboxing to simulate real-world research environments.

Who it’s for

AI researchers and developers who need to benchmark the reasoning and research capabilities of LLMs and agentic frameworks, specifically those focusing on deep-research tasks.

Highlights

• Isolated Execution: Runs each question in a separate subprocess for better stability and reproducibility.
• Broad Benchmark Support: Supports multiple datasets including BrowseComp, DeepSearchQA, and Humanity's Last Exam.
• Tool Integration: Built-in support for web search, web fetch, and code sandboxes.
• Standardized Setup: Implements a standard ReAct pipeline for consistent model evaluation.