grishahq/recursive-llm

Recursive Language Models for efficient long-context processing. Analyze 1M+ tokens by storing context in a Python REPL while reducing LLM token usage.

What it solves

Recursive Language Models (RLM) addresses the problem of analyzing extremely large contexts that are too expensive or too large to fit into a standard LLM prompt. Instead of sending the entire source text to the model, RLM keeps the data in a restricted Python REPL, allowing the model to selectively search, compute, and process only the necessary sections of the text.

How it works

RLM operates by storing the source context as a variable within a persistent Python REPL environment. The root model receives the query and instructions but not the full text. The model then interacts with the context using Python code to perform tasks like regex searches, peeking at specific character ranges, or performing local computations.

When needed, the model can delegate specific tasks to other model calls: it can make a plain LLM call (llm_query) or spawn a child RLM (rlm_query) if the configured recursion depth allows. The process continues iteratively until the model provides a final answer through a specific termination function.

Who it’s for

This library is designed for developers building applications that require high-precision analysis of massive documents where cost and token usage must be bounded. It is particularly useful for tasks that benefit from exact counting, filtering, and local Python computation rather than just semantic synthesis.

Highlights

  • Externalized Context: Keeps large source text in a REPL rather than repeating it in prompts.
  • Bounded Execution: Implements tree-wide limits on the number of calls, tokens, estimated cost, and elapsed time.
  • Provider Portability: Supports over 100 providers via LiteLLM, including OpenAI, Anthropic, DeepSeek, and local models (Ollama, llama.cpp).
  • Restricted Execution: Uses a spawned RestrictedPython worker with hard timeouts and resource limits for security.
  • Observable Runs: Provides detailed per-run statistics, typed failures, and complete execution trajectories in JSONL format.
  • Custom Validation: Allows developers to define a final_answer_validator to reject answers and provide feedback to the model without ending the run.

Related

  • Project
    alexzhang13/rlm

    An inference engine and training environment for Recursive Language Models (RLMs) that enable LLMs to handle near-infinite contexts by programmatically decomposing tasks via recursive sub-calls in a REPL environment.

  • Project
    avbiswas/fast-rlm

    A minimal implementation of Recursive Language Models (RLMs) that allows LLMs to handle arbitrarily long prompts by interacting with data through an external REPL and recursive sub-agents.

  • Project
    Trampoline-AI/predict-rlm

    predict-rlm is a Python library that gives large language models a sandboxed, stateful runtime. Models write and execute Python code, keep mutable state, call focused sub‑models, and use tools/skills (e.g., PDF handling, spreadsheets). The framework records a full trace for observability and can feed those traces to optimisation tools like GEPA. It’s aimed at complex, multi‑step workflows such as document analysis, audit, or spreadsheet automation, and supports optional Docker sandboxing and rich callbacks.

  • Project
    shcherbak-ai/contextgem

    An open-source LLM framework for extracting structured data and insights from documents with precise source references and automatic justifications.

  • Project
    JackHopkins/factorio-learning-environment

    An open-source framework for developing and evaluating LLM agents in Factorio, using a code synthesis REPL pattern to test long-term planning and resource management.