Raudaschl/rag-fusion

RAG‑Fusion: a quick‑read explainer

What it is – RAG‑Fusion is a research‑grade search pipeline that builds on Retrieval‑Augmented Generation (RAG). It tries to overcome the classic “vocabulary mismatch” problem by:

1. Generating alternative queries with a small LLM (four rewrites per user question).
2. Running each query twice – once with a keyword BM25 index and once with a dense‑vector index.
3. Merging the ten result lists using Reciprocal Rank Fusion (RRF), which rewards documents that appear high in many lists.
4. Optional reranking – a cross‑encoder or a decision‑model (e.g., Jev, OpenAI Decisions API) re‑scores the fused pool, after which the top‑5‑10 hits are fed to a generative answer model.

Why it matters – The extra rewrites and hybrid BM25 + vector search add only a modest latency (≈1.5 s for the rewrite call) and a tiny API cost (≈$0.06 per 1 000 queries). In the authors’ experiments on the medical‑focused NFCorpus benchmark, the full Hybrid + Diverse + Rerank configuration lifted NDCG@10 by roughly +0.05 (≈19 % over a plain vector baseline) and MRR by +0.09. The gains are most pronounced when:

• user phrasing differs from the corpus terminology,
• recall is more important than pinpoint precision, and
• the downstream system can handle a short list of retrieved passages (e.g., a synthesis LLM or a UI that shows several candidates).

When to skip it – If you need sub‑second latency, have a tightly curated FAQ‑style knowledge base, or are searching code/identifiers where exact matches dominate, the extra compute rarely pays off.

How to try it – Clone the repo, install the Python deps, add an OpenAI key (and optionally a Jev key), then run:

python main.py          # demo end‑to‑end
python evaluate.py      # run the quantitative benchmark

The eval/ folder contains the full evaluation harness (dataset loader, IR metrics, retrieval variants, rerankers, latency/ cost benchmarks, and bootstrap confidence‑interval utilities). The experiments/ subfolders hold detailed write‑ups and raw results for the various rerankers (Jev, OpenAI Decisions, Microsoft‑Decision‑1) and a replication of an arXiv paper.

Bottom line – RAG‑Fusion is a concrete, reproducible recipe for boosting retrieval quality in low‑to‑moderate‑throughput, knowledge‑intensive search scenarios. It shows that a cheap LLM‑driven query‑expansion step, combined with a free‑lunch BM25 + vector hybrid and a modest reranker, can give statistically significant lifts over standard RAG pipelines.