Sidekick: what it is, what problem it solves & why it's gaining traction

What it solves

Sidekick is a local-first AI assistant for macOS that allows users to chat with LLMs while keeping data secure and offline. It solves the problem of privacy concerns and software installation hurdles by providing a built-in inference engine, enabling users to interact with their own files, folders, and websites without needing external software or cloud-based processing.

How it works

Sidekick uses a built-in llama.cpp backend to run modern GGUF local models (like Qwen 2.5) on Apple Silicon. It employs Retrieval Augmented Generation (RAG) to fetch and reference materials from user-defined "experts" (collections of files and websites) to provide grounded answers with citations. It also supports OpenAI-compatible APIs for those who want to integrate remote models.

Who it’s for

Mac users with Apple Silicon (minimum 8GB RAM) who want a private, context-aware AI assistant that can perform research, analyze local documents, and generate content without relying on the cloud.

Highlights

• Local-First RAG: Accesses files, folders, and websites via configurable "experts" to provide cited answers.
• Agentic Capabilities: Supports function calling for mathematical/logical tasks and a "Deep Research" agent for multi-step, long-horizon research tasks.
• Llama.cpp Backend: Optimized for Apple Silicon with support for speculative decoding for faster generation.
• Multimodal Features: Includes built-in CoreML image generation (macOS 15.2+) and a "Canvas" for editing and previewing code and websites.
• Memory: Remembers preferences and details across conversations for personalized responses.
• Advanced Rendering: Native LaTeX for math and automatic data visualization for tables.