Asana optimizes browser agent costs with GPT-6.1 Sol

Asana has reduced its browser agent model costs by 76x and improved execution speed by 5x by optimizing its workflows on GPT-6.1 Sol. This was achieved by using GPT-6 Astra in Codex to automate the research and experimentation process, reducing a task that would typically take two months of human effort to a single week.

Automated Optimization via GPT-6 Astra

Asana utilized GPT-6 Astra in Codex to identify inefficiencies in its browser agent's codebase and execute a series of controlled experiments. The process involved mapping the codebase to understand how model requests were constructed and testing specific architectural changes to reduce costs and latency.

Inefficiency Identification

GPT-6 Astra discovered two primary bottlenecks in the original agent workflow:

  • Caching Gaps: While fixed instructions and tool definitions were cached, the growing history of page text and screenshots was not, causing every request to resend the entire history at full price.
  • History Volatility: The agent frequently trimmed text and dropped screenshots at nearly every step. These constant edits prevented effective caching and occasionally forced the agent to revisit pages it had already read.

Experimental Design and Execution

Following the identification of these inefficiencies, GPT-6 Astra refactored the agent's code to support parallel workflows with independent settings. It then conducted a 144-run study testing four frontier models (GPT-6.1 Sol and three others labeled Models A, B, and C) across various configurations:

  • History Budgets: Tested budgets of 120,000 and 480,000 characters.
  • Caching and Screenshot Policies: Tested six different policies. The most effective policy allowed screenshots to accumulate to 20 before resetting to the most recent one, which maintained history stability for longer periods.

Performance and Cost Benchmarks

The optimized workflow on GPT-6.1 Sol demonstrated significant improvements in cost and speed compared to the original production setup on Model B.

Cost Reductions

  • Overall Reduction: The optimized workflow on GPT-6.1 Sol averaged $0.47 per run, compared to at least $36.21 per run on the original Model B setup (a 76x reduction).
  • Model-Specific Gains: For Model B, optimization alone reduced costs from $\ge$ $36.21 to $1.24 per run (a 29x reduction). GPT-6.1 Sol was 2.6x cheaper than the optimized Model B.
  • Caching Impact: On GPT-6.1 Sol, the new caching and screenshot policy reduced costs 4x (from $1.97 to $0.47 per run), with 89% of input coming from cache at 5% of the uncached price.

Speed and Reliability

  • Execution Speed: The original setup on Model B took at least 22.5 minutes per run; the optimized workflow on GPT-6.1 Sol took approximately four minutes.
  • Success Rate: Increasing the history budget for GPT-6.1 Sol improved the success rate from 3 out of 18 runs (with the smaller budget) to 18 out of 18 runs (100% success) with the larger budget.

Integration and Future Development

Asana has integrated these optimizations into StackAI and is developing tools to make these experiments repeatable. The company is also leveraging GPT-6 Astra in Codex for pre-release product testing, where the agent navigates the platform to report bugs for human QA reviewers.

"This is what teams of humans and agents look like in practice. An engineer set the direction, GPT-6 Astra ran the experiments, and the results went through Command to production."

— Arnab Bose, CPO at Asana

"Shipping speed is no longer the bottleneck; human attention is. We’re close to a world where every engineer is a PM leading a fleet of agents."

— Frank Hidalgo, PhD, StackAI CTO at Asana

Sources