google-gemini/gemini-live-api-examples

Gemini Live provides multimodal realtime agent capabilities. Build voice agents that can process vision and text in realtime.

What it solves

This repository provides implementation examples for the Gemini Live API, which enables the creation of low-latency, real-time voice and video interactions. It solves the problem of building natural, human-like conversational AI agents that can process continuous streams of audio, video, and text simultaneously.

How it works

The project provides several implementation paths based on the following technical specifications:

  • Protocol: Uses a stateful WebSocket connection (WSS) to maintain a real-time stream.
  • Input/Output: Processes raw 16-bit PCM audio (16kHz input, 24kHz output), JPEG images/video (up to 1FPS), and text.
  • Implementation Examples: Includes a Python backend using the Gen AI SDK, a JavaScript frontend with a Python backend using raw WebSockets, and minimal command-line apps in both Python and Node.js.

Who it’s for

Developers building real-time multimodal agents for industries such as e-commerce (shopping assistants), gaming (interactive NPCs), robotics, healthcare (patient support), and education (AI mentors).

Highlights

  • Multilingual Support: Ability to converse in 70 supported languages.
  • Barge-in: Allows users to interrupt the model for more responsive interactions.
  • Tool Use: Integration with function calling and Google Search for dynamic interactions.
  • Audio Transcriptions: Provides real-time text transcripts of both user and model output.
  • Proactive Audio: Control over when and in what contexts the model responds.

Related

  • Project
    GetStream/Vision-Agents

    A framework for building low-latency, multi-modal AI agents that can watch, listen, and understand video in real time using a combination of CV models and LLMs.

  • Dispatch
    Gemini 2.5 Flash Native Audio release: enhanced live voice agents and real‑time speech translation

    Google DeepMind announced Gemini 2.5 Flash Native Audio, a live‑voice model that improves function calling, instruction adherence, and multi‑turn conversation while adding real‑time speech‑to‑speech translation for over 70 languages.

  • Project
    livekit-examples/python-agents-examples

    A collection of runnable Python examples and full-stack applications for building voice, video, and telephony AI agents using the LiveKit Agents framework.

  • Project
    yeahhe365/AMC-WebUI

    An all-in-one model console WebUI that prioritizes Google Gemini's native capabilities and OpenAI compatibility, featuring a local-first approach to data privacy.

  • Project
    google-gemini/gemini-skills

    A library of context-providing skills for AI coding assistants to help them generate accurate, up-to-date code for the Gemini API and SDKs.