zjunlp/LightMem

[ICLR 2026] LightMem: Lightweight and Efficient Memory-Augmented Generation

What it solves

Large language models (LLMs) and AI agents often lose context across conversations, forcing them to reprocess entire histories. That is slow and expensive. LightMem gives these systems a lightweight long-term memory: it stores, retrieves, and updates previous interactions so that applications can remember relevant details without paying huge token or latency costs.

How it works

LightMem is a modular framework. It compresses raw conversation history with a pre-compressor (like LLMLingua-2), segments it into topics, and generates metadata and summaries. It stores the processed memory in a vector database (Qdrant, FAISS) using embedding models. When you ask a question, LightMem retrieves the most relevant memories and feeds them to the LLM. Memory can also be updated offline by scoring and refining stored entries. The framework supports multiple backends — cloud APIs (OpenAI, DeepSeek) and local models (Ollama, vLLM) — and even offers an MCP server for integration with other tools.

Who it’s for

Developers building AI assistants, agents, or chat applications that need persistent memory. Researchers who want to reproduce experiments on long-context benchmarks like LoCoMo and LongMemEval. Users who want a lightweight, cost-effective alternative to heavier memory systems.

Highlights

  • Lightweight and efficient: minimal resource consumption and fast response times.
  • Easy to use: simple API — integrate with just a few lines of code.
  • Flexible and extensible: modular architecture supports custom storage engines and retrieval strategies.
  • Broad compatibility: works with OpenAI, DeepSeek, Ollama, vLLM, and Transformers.
  • Provides reproduction scripts for LoCoMo and LongMemEval benchmarks with strong accuracy and low token usage.
  • Includes an MCP server for easy integration with other tools.
  • Offers tutorial notebooks for travel planning, code assistant, and long-memory evaluation.

Related

  • Project
    TIMAN-group/PlugMem

    A plug-and-play long-term memory system for LLM agents that organizes experience into a graph of compact, reusable knowledge units to reduce context overhead.

  • Project
    aiming-lab/SimpleMem

    A unified memory stack for LLM agents that provides semantically lossless compression for text and multimodal data, featuring a self-evolving retrieval system to maximize recall while minimizing token usage.

  • Project
    VictorTaelin/OptMem

    A permanent memory system for AI agents that uses a binary tree of summaries and a CLI tool to maintain persistent knowledge across sessions.

  • Project
    ModelTC/LightLLM

    LightLLM is a Python‑based, high‑performance inference and serving framework for large language models. It combines fast kernels (FlashAttention, Triton) with lightweight, token‑level KV‑cache management, supports multi‑GPU scaling, and adds research‑grade features like constrained decoding and SLA‑aware request scheduling.

  • Project
    sopaco/cortex-mem

    A Rust-based long-term memory framework for AI agents that uses a three-tier hierarchical architecture and a virtual filesystem to provide persistent, context-aware interactions.