avatarin Retail Agent powered by GPT-Realtime
avatarin has developed a 24/7 multilingual retail agent for Yamada Holdings using OpenAI’s GPT-Realtime, enabling natural voice conversations that guide shoppers from product discovery to purchase. In a two-week public campaign, the agent served approximately 30,000 users with a 92% positive survey response rate.
Multimodal Integration via GPT-Realtime
GPT-Realtime allows the Kurashi-Marugoto AI Agent to integrate voice, text, and visual understanding into a single, low-latency conversational experience. Unlike conventional chatbots that rely on keyword triggers, this system adapts to the way people communicate, responding to context, uncertainty, and changing preferences.
According to Akira Fukabori, CEO of avatarin, the performance of a single multimodal model exceeds that of specialized speech recognition systems, allowing the agent to handle complex queries—such as finding a refrigerator for a family of four in a small kitchen—with human-like intelligence rather than rigid machine rules.
Technical Implementation and Design Principles
To ensure the agent functioned as an expert sales associate, avatarin focused on three core technical and design pillars:
Grounding and Responsiveness
To maintain accuracy without sacrificing speed, the agent utilizes a retrieval-augmented generation (RAG) system. This grounds responses in actual product information while GPT-Realtime ensures the interaction remains fluid and responsive.
Incorporation of Domain Expertise
avatarin integrated Yamada Denki’s specific sales knowledge into the agent's prompting and conversation flows. The system is designed to handle temporary off-topic diversions and changing customer requirements while maintaining guardrails to keep the interaction focused on the shopping experience.
Proactive Guided Discovery
Rather than acting as a passive Q&A tool, the agent is designed to be proactive. It asks follow-up questions to uncover a customer's actual needs, shifting the experience from a simple inquiry to guided product discovery.
OpenAI supported this implementation by helping avatarin structure complex prompts, optimize API costs for always-on voice services, and share best practices.
Performance Metrics and Business Insights
During a two-week public campaign on Yamada Denki’s online store, the agent achieved the following results:
- User Reach: Approximately 30,000 users.
- Availability: 24/7 multilingual support via voice and text.
- Customer Satisfaction: 92% positive survey responses.
Beyond these metrics, the agent provided deep qualitative insights into shopper behavior. Because the interactions were conversational, Yamada Holdings could identify why shoppers hesitated and what specific factors influenced their decisions—data that is typically difficult to capture in conventional online shopping.
Vision for Omnichannel AI
avatarin aims to move beyond single-purpose AI toward a model of "One Intelligence. One Brand. Every interface." This vision involves a single AI agent that maintains customer context seamlessly across the web, phone, and physical stores. For Yamada Holdings, this means the AI can embody the company's brand and hospitality across its various business sectors, including home appliances, housing, and financial services.