addyosmani/chatty

ChattyUI - your private AI chat for running LLMs in the browser

What it solves

Chatty provides a private, offline-capable AI chat experience that runs entirely within the web browser. It eliminates the need for server-side processing, ensuring that user data and uploaded documents remain on the local machine.

How it works

The application leverages WebGPU to run large language models (LLMs) natively on the user's hardware. It integrates WebLLM for model execution and uses XenovaTransformerEmbeddings and MemoryVectorStore to enable local RAG (Retrieval-Augmented Generation), allowing users to chat with uploaded PDF and text files without sending them to a cloud server.

Who it’s for

Users who want a ChatGPT-like interface but require strict data privacy, offline access, or the ability to run open-source models (like Llama 3, Gemma, and Mistral) directly in their browser.

Highlights

  • Local Execution: Runs entirely client-side via WebGPU for maximum privacy.
  • Local File Chat: Supports uploading PDFs and code files for local querying.
  • Offline Mode: Works without an internet connection after the initial model download.
  • Broad Model Support: Compatible with popular open-source models including Llama 2, Llama 3, Gemma, and Mistral.
  • Rich UI: Includes markdown rendering, code highlighting, voice input, and light/dark modes.

Related

  • Project
    Ryan-yang125/ChatLLM-Web

    A private, browser-native AI studio and agent workspace that runs LLMs locally on your device using WebGPU, requiring no API keys or backend servers.

  • Project
    ServiceStack/llms

    A lightweight CLI, API, and web interface for accessing multiple LLMs privately, with all data stored locally in the browser.

  • Project
    mlc-ai/web-llm

    A high-performance in-browser LLM inference engine that uses WebGPU to run language models locally without a server.

  • Project
    Vali-98/ChatterUI

    A native mobile frontend for LLMs that allows users to run models on-device using llama.cpp or connect to various commercial and open-source APIs.

  • Project
    nico-martin/gemma4-browser-extension

    A Chrome extension that runs the Gemma 4 model locally via WebGPU to provide a private AI assistant capable of browser tab management and semantic webpage interaction.