xAI Voice Agent Builder Beta Release

xAI has introduced the Voice Agent Builder in beta, a no-code platform designed to allow developers and operators to configure production voice agents using Grok Voice. The platform eliminates the need to build a complex surrounding stack from scratch by integrating telephony, knowledge retrieval, tools, guardrails, and observability into a single interface.

Native Speech-to-Speech Architecture

Voice Agent Builder utilizes a speech-to-speech path built for Grok Voice, rather than the traditional method of stitching together three separate APIs for speech-to-text, a language model, and text-to-speech. By tightly coupling the model to the voice path, xAI aims to reduce the cost, latency, and failure modes associated with multi-provider API hops.

Performance Benchmarks on Complex Calls

To address the challenges of real-world telephony—such as low-quality audio, background noise, strong accents, and interruptions—Grok Voice was trained on high-difficulty calls. Performance is measured via the τ-voice Bench, which simulates these conditions. According to the xAI leaderboard, Grok Voice Think Fast 1.0 achieves a score of 67.3%, significantly outperforming Gemini 3.1 Flash Live (43.8%) and GPT Realtime 1.5 (35.3%).

Agent Configuration and Capabilities

Users can deploy a working agent in approximately two minutes by providing a plain-language description of call flows and attaching necessary resources.

Knowledge Base and Retrieval

Agents are grounded in a knowledge base where users can upload documents in formats including Markdown, plain text, Word, PowerPoint, Excel, HTML, and JSON. These documents are organized into collections, allowing policies and product specifications to be shared across multiple agents to ensure consistency.

Tool Integration and Actionability

Agents can perform actions through tools and connectors, enabling them to:

• Schedule appointments via Google Calendar or Outlook Calendar.
• Send confirmations through email providers.
• Check order statuses or issue refunds via API requests.
• Retrieve current public information using web search or X search.
• Manage tickets in Linear or Notion.
• Access files from Google Drive or OneDrive.
• Transfer calls to human team members or end calls cleanly.

Voice Customization and Telephony

Agents can utilize built-in voices or a custom brand voice clone created from two minutes of audio. Connectivity options include a free provisioned phone number, direct SIP connection for existing numbers from major telephony providers, or browser-based testing via WebSocket.

Observability and Guardrails

Every call is recorded and transcribed, allowing operators to review audio, read transcripts, and inspect tool usage. The platform includes guardrails to prevent agents from performing prohibited actions, such as discussing off-script topics or reading back credit card numbers.

Pricing Model

Voice Agent Builder is billed at the API rate of $0.05 per minute of audio, with voices included and no separate platform fee. Telephony on a free provisioned number costs an additional $0.01 per minute.