google/xrblocks

XR Blocks is a lightweight WebXR + AI library for rapidly prototyping advanced AI + XR experiences.

What it solves

XR Blocks simplifies the complex process of building immersive XR (Extended Reality) applications that integrate AI. It provides a unified, cross-platform framework that allows developers to create interactive 3D experiences that run on both specialized XR hardware and standard desktop browsers without needing separate codebases.

How it works

Built on top of the Three.js library, XR Blocks provides a high-level SDK that abstracts away the difficulties of handling spatial input and AI integration. It includes a built-in desktop simulator for rapid testing, handles hand and head gesture recognition, and provides a spatial UI system. For AI features, it connects directly to Gemini for multimodal understanding, allowing developers to build conversational, context-aware XR applications.

Who it’s for

This library is designed for developers and researchers looking to prototype and build AI-native XR applications. It is particularly useful for those who want to leverage AI agents, gesture-based interactions, and spatial computing without managing low-level hardware APIs.

Highlights

  • Cross-platform support for Android XR devices and desktop Chrome browsers.
  • Advanced gesture recognition for hands (pinch, palm, fist, etc.) and head movements (nod, shake).
  • Built-in spatial UI components like buttons, sliders, and flex-layout panels.
  • Seamless integration with Gemini for conversational AI and multimodal scene understanding.
  • Includes a desktop simulator to test XR interactions without needing a headset.
  • Provides a dedicated benchmark (GenXR-Bench) for evaluating AI coding agents in XR development.

Related

  • Project
    yeahhe365/Gemini-Nexus

    Gemini Nexus is a Chrome MV3 extension that adds a native AI assistant to the browser. It supports Google Gemini (both the web UI via reverse‑engineered tokens and the official API) and many OpenAI‑compatible services. The extension provides a side‑panel, a floating selection toolbar, image/screenshot input, and a Chrome‑DevTools‑based control loop so the LLM can navigate pages, click elements, fill forms, run scripts, and call external tool‑servers (MCP). It includes context‑management features, watermark removal for Gemini images, and a full build/publish workflow. The Gemini‑Web integration may violate Google’s Terms of Service, so users must accept the associated risk.

  • Project
    google-gemini/gemini-cli

    Gemini CLI is an open‑source Google tool that brings Gemini LLMs to the terminal, enabling code generation, debugging, and automation via an interactive REPL or scripted commands, with multiple auth methods and extensible MCP‑server integrations.

  • Project
    googleapis/js-genai

    Official Google npm SDK (`@google/genai`) that wraps Gemini’s public and Enterprise Agent APIs for JavaScript/TypeScript. Provides simple classes for one‑shot generation, streaming, stateful chats, file uploads, live sessions, and a unified Interactions API. Supports function/tool calling, automatic continuation, and multimodal inputs. Install with `npm i @google/genai`; use an API key or enterprise credentials. Documentation and samples are hosted on the repo’s website.

  • Project
    google/agents-cli

    agents‑cli is a Google‑maintained CLI (and skill bundle for AI coding assistants) that streamlines the entire lifecycle of Gemini‑based AI agents—from project scaffolding and local evaluation to cloud deployment, CI/CD, and observability.

  • Project
    simonw/llm-gemini

    A plugin for the llm CLI tool that enables seamless access to Google's Gemini models, including support for multimodal inputs, Google Search grounding, code execution, and vector embeddings.