johnbean393/Sidekick

A native macOS app that allows users to chat with a local LLM that can respond with information from files, folders and websites on your Mac without installing any other software. Powered by llama.cpp.

What it solves

Sidekick is a local-first AI assistant for macOS that allows users to chat with LLMs while keeping data secure and offline. It solves the problem of privacy concerns associated with cloud AI, providing a way to integrate personal files, folders, and websites as context for the AI without needing to install separate inference software.

How it works

Sidekick uses a built-in llama.cpp backend to run GGUF local models (like Qwen 2.5) directly on Apple Silicon. It employs Retrieval Augmented Generation (RAG) to fetch relevant information from user-defined "experts" (collections of files and websites) to answer questions with direct quotes and references. It also supports OpenAI-compatible APIs for those who want to use remote models.

Who it’s for

Mac users who want a private, context-aware AI assistant that can research documents, automate tasks via function calling, and generate images locally.

Highlights

  • Local-First Inference: Built-in llama.cpp for fast generation on Apple Silicon, including support for speculative decoding.
  • RAG-powered Experts: Organizes resources into thematic collections that the AI can reference with page numbers and citations.
  • Deep Research Agent: A specialized agent capable of reading dozens of webpages to synthesize comprehensive research reports.
  • Function Calling: Ability to execute local actions, such as drafting emails or calculating financial metrics from CSVs.
  • Canvas: An interactive area to create, edit, and preview websites and code with selective editing capabilities.
  • Multimodal Capabilities: Automatic image generation via CoreML (on macOS 15.2+) and native LaTeX and data visualization rendering.
  • Persistent Memory: Remembers user preferences and project details across different conversations.

Related

  • Project
    google-ai-edge/gallery

    A mobile and macOS application for running open-source LLMs and multimodal AI models fully on-device, providing a private and offline experience for chat, media search, and audio transcription.

  • Project
    kansoku-trade/kansoku

    A local-first macOS AI trading assistant that provides verifiable market analysis and technical indicators while keeping all data and API keys on the user's machine.

  • Project
    MindWorkAI/AI-Studio

    MindWork AI Studio is a free cross‑platform desktop app that lets users chat with many LLM providers (cloud or local), use pre‑built assistants, run tool‑enabled RAG, and manage settings centrally for enterprises. It runs on low‑cost hardware, supports voice transcription, and is released under a functional source license that becomes MIT after two years.

  • Project
    aws-samples/amazon-bedrock-client-for-mac

    A native Swift application for macOS that provides a direct client for Amazon Bedrock, enabling local storage of conversations and integration with local system tools.

  • Project
    Capsize-Games/airunner

    AI Runner is a self‑hosted desktop app that offers a voice‑enabled AI companion with long‑term memory and a layered canvas for local AI‑generated art. All inference (LLM, diffusion, TTS/STT) runs on the user’s machine, with optional Docker or headless API deployments.