google-gemini/gemini-live-api-examples

Gemini Live provides multimodal realtime agent capabilities. Build voice agents that can process vision and text in realtime.

What it solves

This repository provides implementation examples for the Gemini Live API, which enables the creation of low-latency, real-time voice and video interactions. It solves the problem of building natural, human-like conversational experiences that can process continuous streams of audio, video, and text simultaneously.

How it works

The project demonstrates how to connect to the Live API using a stateful WebSocket connection (WSS). It provides several implementation paths:

  • Gen AI SDK: A Python-based approach for ease of use.
  • Raw WebSockets: A JavaScript frontend and Python backend for direct protocol control.
  • Command-line tools: Minimal apps in Python and Node.js for streaming microphone audio and receiving real-time responses.
  • Translation tools: Examples of streaming remote audio URLs for real-time translation.

Who it’s for

Developers building real-time AI agents for industries such as e-commerce (shopping assistants), gaming (interactive NPCs), healthcare (health companions), education (AI mentors), and next-gen interfaces for robotics or smart glasses.

Highlights

  • Multimodal Input/Output: Supports raw 16-bit PCM audio, JPEG images/video, and text.
  • Barge-in: Allows users to interrupt the model during responses for more responsive interactions.
  • Tool Use: Integrates function calling and Google Search for dynamic capabilities.
  • Affective Dialog: Adapts the tone and style of responses to match the user's expression.
  • Linguistic Diversity: Support for 70 languages.
  • Audio Transcriptions: Provides text transcripts for both user and model output.

Related

  • Project
    GetStream/Vision-Agents

    A framework for building low-latency, multi-modal AI agents that can watch, listen, and understand video in real time using a combination of CV models and LLMs.

  • Dispatch
    Gemini 2.5 Flash Native Audio release: enhanced live voice agents and real‑time speech translation

    Google DeepMind announced Gemini 2.5 Flash Native Audio, a live‑voice model that improves function calling, instruction adherence, and multi‑turn conversation while adding real‑time speech‑to‑speech translation for over 70 languages.

  • Project
    livekit-examples/python-agents-examples

    A collection of runnable Python examples and full-stack applications for building voice, video, and telephony AI agents using the LiveKit Agents framework.

  • Project
    pipecat-ai/pipecat-examples

    A collection of intermediate and advanced example applications demonstrating how to build voice and multimodal AI agents using the Pipecat framework.

  • Project
    TEN-framework/ten-framework

    An open-source framework for building real-time multimodal conversational AI agents with low-latency audio, text, and visual capabilities.