shinpr/mcp-local-rag

Local-first RAG server for developers. Semantic + keyword search for code and technical docs. Works with MCP or CLI. Fully private, zero setup.

What it solves

mcp-local-rag is a local Retrieval-Augmented Generation (RAG) system that allows users to search private documents without sending data to external embedding APIs. It addresses the need for confidentiality and cost-efficiency by keeping all document parsing, embeddings, and storage on the machine. It also solves the problem of semantic search missing exact technical terms (like API names or error codes) by implementing a hybrid search approach.

How it works

The system operates in two main phases:

  1. Ingestion: It parses supported files (PDF, DOCX, Markdown, TXT) or HTML data, splits them into chunks using semantic chunking (preserving Markdown code blocks), and generates embeddings locally using Transformers.js. These are stored in a LanceDB vector database.
  2. Search: When a query is made, the system embeds the query and performs a vector search for semantically related chunks. It then applies a keyword boost to ensure exact technical terms are ranked higher.

Additionally, it offers an optional "visual mode" for PDFs, which uses a local vision model to generate searchable captions for figures and tables.

Who it’s for

  • Developers and researchers who need to search through technical documentation or private papers.
  • Users of AI coding tools (such as Claude Code, Cursor, or Codex) who want to integrate local document search via the Model Context Protocol (MCP).
  • Privacy-conscious users who cannot send sensitive documents to hosted embedding services.

Highlights

  • Fully Local: No API keys, Docker, or external databases required; works offline after initial model download.
  • Hybrid Search: Combines semantic retrieval with keyword matching for better technical accuracy.
  • MCP Integration: Works as an MCP server, allowing AI agents to sync, ingest, and query documents directly.
  • Semantic Chunking: Splits documents at topic boundaries rather than fixed character counts.
  • Visual PDF Indexing: Optional local VLM support to create searchable captions for PDF figures.

Related

  • Project
    PromtEngineer/localGPT

    A private, on-premise document intelligence platform that uses hybrid search and local LLMs to let users query and summarize their files without data leaving their machine.

  • Project
    andrea9293/mcp-documentation-server

    A local-first document management and semantic search server for AI agents that provides a private knowledge base with a built-in web UI and hybrid search.

  • Project
    tonykipkemboi/ollama_pdf_rag

    A local RAG application that lets users chat with multiple PDF documents using Ollama and LangChain, featuring Next.js and Streamlit interfaces.

  • Project
    danny-avila/rag_api

    An asynchronous FastAPI-based RAG API that uses pgvector or MongoDB to provide ID-based document indexing and retrieval with built-in ownership security.

  • Project
    rag-web-ui/rag-web-ui

    An intelligent dialogue system that allows users to build custom Q&A services by combining their own document knowledge bases with LLMs using Retrieval-Augmented Generation.