OpenAI GPT-6 Astra achieves 32% higher performance on Ironclad contracting tasks
TL;DR
OpenAI released GPT‑6 Astra, a frontier language model fine‑tuned on Ironclad’s contracting workflows, and showed it scores 32 % higher than GPT‑5.6 Sol on a suite of 11 real‑world tasks while completing attempts 48 % faster.
Partnership Overview
OpenAI is collaborating with a select group of software companies to turn high‑value, domain‑specific workflows into research problems. Ironclad, a leader in AI‑assisted contracting, is the first partner. Together they defined concrete tasks, supplied hosted environments for model practice, and established detailed evaluation criteria.
"Ironclad’s expertise has been instrumental in defining what success looks … and bringing real customer needs directly into frontier model development." – OpenAI announcement
Translating Contracting Workflows into Training Tasks
Legal operations use cases – The team identified 11 representative tasks spanning legal, commercial, and procurement work (e.g., configuring nondisclosure agreements, building procurement approval processes, updating reusable clauses based on jurisdiction). An experienced human typically spends 30–40 minutes per task.
Task design – Each task was evaluated against 8–50 criteria, allowing granular insight into model strengths and weaknesses. Ironclad provided sandboxed instances of their product so models could execute the tasks directly.
Training pipeline – Researchers generated synthetic variants of the workflows and applied reinforcement learning from human feedback (RLHF) to let the model practice repeatedly and receive corrective signals.
Performance of GPT‑6 Astra
| Model | Avg. Score | Avg. Time per Attempt |
|---|---|---|
| GPT‑6 Astra (Max reasoning) | 55.0 % | 19.2 min |
| GPT‑5.6 Sol (High reasoning) | 41.6 % | 37.0 min |
| Internal Astra prototype | 63.7 % | – |
Astra’s score is 32 % higher than Sol’s (55.0 % vs. 41.6 %) and its estimated time per attempt is 48 % lower (19.2 min vs. 37.0 min). In a side‑by‑side video demo, Astra satisfied roughly 94 % of task criteria in ~20 minutes, whereas Sol satisfied about 85 % in ~32 minutes.
Implications for Ironclad and Business Software
Rule fidelity – Contracting platforms must enforce business rules across many exception paths. Astra’s ability to maintain those rules throughout multi‑step workflows suggests agents can increasingly handle high‑stakes professional tasks, though human oversight remains essential.
Product opportunities – As models improve, Ironclad can embed them across its suite to automate repetitive contract configuration while preserving the governance controls that legal teams require.
Call for Additional Partnerships
OpenAI invites a limited number of software companies to join this research program. Desired partners must provide:
- A concrete, high‑impact professional task that current agents fail to complete reliably.
- Clear failure examples and success metrics.
- Domain experts and a secure testing environment.
- Data that can be safely used for research.
These collaborations aim to surface real‑world failure modes, measure improvements, and accelerate the deployment of capable AI agents in complex professional domains.
All performance numbers are drawn from OpenAI’s internal research evaluation and have not been independently audited.