Cloudflare Web Search API Beta

Cloudflare has introduced the Web Search API in beta, enabling AI agents and applications to retrieve live internet information to ground their responses. This tool is designed to eliminate the common AI issues of "hallucinating" URLs or relying on outdated training data by providing a standardized interface for real-time web retrieval.

Integrated Search Providers and Pricing

Cloudflare does not maintain its own search index; instead, it acts as a gateway to three third-party providers. Users can utilize Cloudflare's AI Gateway credits or provide their own API keys.

Available Providers

• Ceramic.ai: The most affordable option at approximately $0.25 per 1,000 requests.
• Linkup: An EU-based provider guaranteeing GDPR compliance, priced at approximately $5.00 per 1,000 requests.
• Exa: A high-end provider priced at approximately $7.00 per 1,000 requests.

Implementation Methods

Developers can integrate the API via a standard REST call or directly within a Cloudflare Worker using the AI binding:

REST API Example:

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/websearch/ \
  --request POST \
  --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "query": "What are some fun things to do in Salt Lake City as fall approaches?",
    "provider": "ceramic",
    "limit": 5,
    "options": { "gateway": { "id": "default" } }
  }'

Cloudflare Worker Example:

const response = await env.AI.websearch({
	gatewayId: "default",
	query: "What are some fun things to do in Salt Lake City as fall approaches?",
	provider: "exa",
	limit: 5,
});

const results = await response.json();

Data Privacy and Retention

Cloudflare states that all three providers support Zero Data Retention (ZDR) for requests made through Cloudflare. However, this claim has been met with skepticism by the developer community.

• Conflicting Documentation: Users noted that Cloudflare's own provider page lists Exa as not supporting Zero Data Retention, contradicting the main announcement.
• Privacy Policy Concerns: Some users pointed out that Exa's general privacy policy mentions training on data, raising questions about how ZDR is implemented specifically for Cloudflare traffic.
• Logging: Because the API runs through AI Gateway, search requests are recorded in gateway logs, leading to questions about whether billing data can be separated from the actual search queries to maintain true privacy.

Community Analysis and Critical Perspectives

Technical discussion on Hacker News reveals several points of contention regarding the value proposition and ethics of the Web Search API.

The "Gatekeeper" Paradox

A significant portion of the community expressed concern over Cloudflare's dual role as both a bot-protection service and a provider of search APIs.

"Create bot detection and bot protection, then sell crawlers. Is this the peak of hypocrisy?"

Critics argue that Cloudflare may be creating a "pay-to-play" ecosystem where only "verified bots" from their partner providers can bypass the bot protection that Cloudflare sells to website owners.

Value Proposition vs. Direct Integration

Many developers questioned the utility of adding Cloudflare as a proxy layer between the application and the search provider.

• Latency and Complexity: Some argue that a "cloud hop" adds unnecessary latency and a new dependency without providing significant abstraction or normalization of the data returned by different providers.
• Cost: Users noted that some alternatives, such as Gemini Flash Lite 2.5 (which offers 1,000 free searches per day) or Serper.dev ($1 per 1,000), may be more cost-effective than the providers integrated into Cloudflare.
• Terms of Service: Developers warned that search APIs often have restrictive terms. For example, Ceramic's terms reportedly prohibit the collection, aggregation, or storage of output to create a database or index, which could limit the ability of agent systems to provide "share transcript" features.

Performance Feedback

Early anecdotal reports suggest varying quality. One user reported that Ceramic.ai failed to return relevant results for specific technical queries (e.g., "qwen-3.8 flash next") that were easily found on Google or DuckDuckGo.