zjunlp/LightMem

[ICLR 2026] LightMem: Lightweight and Efficient Memory-Augmented Generation

What it solves

LightMem is designed to provide Large Language Models (LLMs) and AI agents with a lightweight and efficient long-term memory management system. It addresses the challenge of maintaining a persistent, retrievable memory of user interactions and events without consuming excessive computational resources or response time.

How it works

LightMem uses a modular architecture to handle memory storage, retrieval, and updates. It breaks the process down into pluggable components:

  • Pre-compression & Segmentation: Uses tools like llmlingua-2 for compressing input and segmenting text by topic.
  • Memory Management: Integrates with various LLM backends (OpenAI, DeepSeek, Ollama, vLLM) to manage memory content.
  • Indexing & Retrieval: Employs embedding-based retrieval (via FAISS, Qdrant, or BM25) to find relevant memories based on user queries.
  • Update Mechanism: Supports both online and offline updates to keep the memory current.

Who it’s for

This framework is intended for developers building intelligent applications, such as travel planning agents or code assistants, that require long-term memory capabilities across multiple sessions.

Highlights

  • Modular Design: Supports custom storage engines and retrieval strategies.
  • Broad Compatibility: Works with cloud APIs (OpenAI, DeepSeek) and local models (Ollama, vLLM, Transformers).
  • MCP Server Support: Compatible with the Model Context Protocol for easier integration into AI ecosystems.
  • Efficient Performance: Optimized for low resource consumption and fast response times.
  • Integrated Evaluation: Includes a baseline evaluation framework to benchmark against other memory layers like Mem0 and LangMem.