microsoft/KBLaM

What it solves

KBLaM addresses the challenge of augmenting Large Language Models (LLMs) with external knowledge without the drawbacks of traditional methods. It avoids the need for external retrieval modules (unlike RAG) and prevents the quadratic computational overhead associated with large contexts (unlike in-context learning), allowing overhead to scale linearly with the size of the knowledge base.

How it works

The system uses a base LLM combined with trained adapters. These adapters transform a knowledge base into special knowledge tokens that the LLM can ingest. Because only the adapters are trained, the base LLM remains unmodified; if no knowledge base is provided, the model behaves exactly like the original base model.

Who it’s for

This project is primarily intended for researchers looking for efficient ways to ground LLM responses in external knowledge bases.

Highlights

• Linear Scaling: Computational overhead scales linearly with the size of the knowledge base.
• No External Retrieval: Eliminates the need for separate retrieval modules.
• Non-Destructive: The base LLM is unmodified, preserving its original text-input capabilities.
• Broad Model Support: Compatible with Llama 3, Llama 3.2, and Phi-3-mini.