NVIDIA AI-Q Blueprint: Top-Ranking Open Deep Research Agent on DeepResearch Bench

NVIDIA's AI-Q Blueprint is a portable, open deep research agent that has reached the top of the Hugging Face "LLM with Search" leaderboard on DeepResearch Bench. This achievement demonstrates that open-source AI stacks can power advanced agentic workflows that rival or surpass closed-source alternatives.

Core Technical Stack and Architecture

The AI-Q Blueprint fuses two high-performance open LLMs to manage long-context retrieval, agentic reasoning, and synthesis:

  • Llama 3.3-70B Instruct: Used as the foundation for generating fluent and structured reports.
  • Llama-3.3-Nemotron-Super-49B-v1.5: A reasoning-focused variant optimized via Neural Architecture Search (NAS), knowledge distillation, and supervised and reinforcement learning. It is specifically designed for multi-step reasoning, query planning, tool use, and reflection.

To support these models, the reference example incorporates the NVIDIA NeMo Retriever for scalable multimodal search across internal and external data, and the NVIDIA NeMo Agent toolkit for orchestrating complex, multistep workflows. This architecture allows for parallel, low-latency search over local and web data, facilitating deployment in environments requiring high privacy and compliance.

Deep Reasoning Capabilities of Llama Nemotron

Llama-3.3-Nemotron-Super-49B-v1.5 is post-trained for explicit agentic reasoning and allows users to toggle reasoning ON or OFF via system prompts. This enables the model to function either as a standard chat LLM or as a deep, chain-of-thought reasoning engine for agent pipelines.

Key technical specifications include:

  • Post-training: A multi-phase approach combining instruction following, mathematical/programmatic reasoning, and tool-calling skills.
  • Efficiency: With 49B parameters and a context window of up to 128K tokens, the model can run on a single H100 GPU or smaller, ensuring predictable and fast inference costs.
  • Lineage: The model is directly traceable from open Meta weights, with transparency regarding synthetic data and tuning datasets.

Evaluation Metrics and Transparency

AI-Q emphasizes transparency in reasoning traces and intermediate steps. The development team utilized a combination of standard and specialized metrics to ensure robustness:

  • Hallucination Detection: Factual claims are checked during the generation process.
  • Multi-source Synthesis: The ability to synthesize new insights from disparate evidence.
  • Citation Trustworthiness: Automated assessment of the links between claims and evidence.
  • RAGAS Metrics: Automated scoring for retrieval-augmented generation accuracy.

DeepResearch Bench Performance

DeepResearch Bench evaluates agent stacks using over 100 long-context, real-world research tasks across domains such as science, finance, art, history, and software. These tasks require complex multi-hop reasoning and report-length synthesis rather than simple QA.

As of August 2025, AI-Q achieved an overall score of 40.52 in the LLM with Search category, making it the top-ranking fully open-licensed stack. The system's strongest performance metrics were in comprehensiveness (report depth), insightfulness (quality of analysis), and citation quality.

Availability for Developers

Both Llama-3.3-Nemotron-Super-49B-v1.5 and Llama 3.3-70B Instruct are available for download on Hugging Face. Developers can integrate these models into their own pipelines using Python or deploy them with vLLM for fast inference and tool-calling support.

Sources