jonfairbanks/local-rag

Ingest files for retrieval augmented generation (RAG) with open-source Large Language Models (LLMs), all without 3rd parties or sensitive data leaving your network.

What it solves

It provides a way to perform retrieval augmented generation (RAG) entirely offline. This ensures that chat history, embeddings, and indexed source content remain on the user's own machine or network, avoiding the need to send data to external cloud services.

How it works

The system uses LlamaIndex for streaming RAG responses and integrates with local Ollama models for both chat and embedding generation. It allows users to ingest data from various sources, including local files, GitHub repositories, and websites, which are then indexed for retrieval.

Who it’s for

Users who want to run a private, open-source RAG system on their own hardware to interact with their own documents and data without relying on cloud-based AI.

Highlights

  • Local Ollama chat and Hugging Face embedding models
  • Support for multiple ingestion sources: local files, GitHub repositories, and websites
  • Streaming responses via LlamaIndex
  • Browser-local settings persistence
  • Chat history export
  • Built-in guardrails for uploads, URLs, and repository ingestion