OpenAI New Tools for Building Agents

OpenAI has launched a new set of APIs and tools designed to simplify the development of reliable, production-ready AI agents. These tools address common developer challenges such as extensive prompt iteration and complex custom orchestration logic by providing standardized building blocks for agentic applications.

The Responses API: A New Agent Primitive

The Responses API is a new API primitive that combines the simplicity of the Chat Completions API with the tool-use capabilities of the Assistants API. It allows developers to solve complex tasks using multiple tools and model turns within a single API call.

Key Features and Integration

  • Unified Design: The API features an item-based design, simpler polymorphism, and intuitive streaming events.
  • Developer Helpers: Includes SDK helpers like response.output_text for easier access to model text output.
  • Data Storage: Supports storing data on OpenAI to facilitate performance evaluation through tracing and evaluations, while maintaining that business data is not used for model training by default.
  • API Relationship: The Responses API is a superset of the Chat Completions API. While Chat Completions remains supported for those not requiring built-in tools, OpenAI recommends the Responses API for new integrations.

Transition from Assistants API

OpenAI is working toward full feature parity between the Responses and Assistants APIs, including support for Code Interpreter and Thread-like objects. Once parity is achieved, the Assistants API will be formally deprecated with a target sunset date in mid-2026.

Built-in Agent Tools

The Responses API introduces three primary built-in tools to connect models to real-world data and interfaces.

Web Search

Web search provides fast, up-to-date answers with inline citations. It is available for gpt-4o and gpt-4o-mini and is powered by the same model used for ChatGPT search.

  • Performance: On the SimpleQA benchmark, GPT-4o search preview scored 90% and GPT-4o mini search preview scored 88%.
  • Availability: Available in preview in the Responses API, with direct access to fine-tuned search models via gpt-4o-search-preview and gpt-4o-mini-search-preview in the Chat Completions API.

File Search

The file search tool enables retrieval of information from large document volumes with support for multiple file types, query optimization, metadata filtering, and custom reranking.

  • Use Case: Navan uses file search to provide precise answers from company travel policy knowledge-base articles via a RAG pipeline.
  • Pricing: Priced at $2.50 per thousand queries and file storage at $0.10/GB/day (first GB free).

Computer Use

Powered by the Computer-Using Agent (CUA) model, this tool allows agents to capture mouse and keyboard actions to automate computer tasks.

  • Benchmarks: The CUA model achieved 38.1% success on OSWorld, 58.1% on WebArena, and 87% on WebVoyager.
  • Safety and Reliability: Due to a 38.1% success rate on OSWorld, OpenAI recommends human oversight for non-browser environments. Safety mitigations include guards against prompt injections, confirmation prompts for sensitive tasks, and environment isolation tools.

Agents SDK for Orchestration

OpenAI has released an open-source Agents SDK to simplify the orchestration of single-agent and multi-agent workflows. This SDK is an evolution of the experimental Swarm SDK.

Core Capabilities

  • Agents: Configurable LLMs with specific instructions and tools.
  • Handoffs: Mechanisms to intelligently transfer control between agents.
  • Guardrails: Configurable safety checks for input and output validation.
  • Observability: Integrated tracing tools to visualize and debug agent execution.

Implementation Examples

  • Coinbase: Used the SDK to build AgentKit, allowing AI agents to interact with crypto wallets and on-chain activities.
  • Box: Created agents that combine web search with internal unstructured data to provide secure, permission-aware insights for enterprises.

Compatibility

The SDK supports the Responses API and Chat Completions API, and is compatible with other providers' models that offer a Chat Completions style API endpoint. It is currently available for Python, with Node.js support planned.

Sources