avbiswas/fast-rlm

A feature rich implementation of Recursive Language Models, with ACP integration, REPL tool support, structured IO, advanced visualization, logging tools.

What it solves

fast-rlm implements Recursive Language Models (RLMs), an inference technique designed to handle prompts that are too large for a model's context window. Instead of fitting everything into one call, the agent programmatically explores and transforms data through an external REPL, allowing it to process arbitrarily long contexts without manual splitting.

How it works

The system uses Deno and Pyodide to provide the LLM with a Python REPL. The root agent can write code to search, filter, and decompose tasks. It can recursively spawn sub-agents to handle smaller subtasks, where the sub-agent's responses are returned as variables or symbols within the parent's REPL rather than being automatically loaded into the parent's context window.

Who it’s for

Developers building AI agents that need to process massive datasets (e.g., millions of tokens of transcripts), perform complex data aggregation, or integrate with external tools and MCP servers while maintaining a bounded context window.

Highlights

  • Arbitrarily Long Context: Agents explore data programmatically via REPL instead of relying on context window limits.
  • Recursive Sub-agents: Ability to spawn child agents to decompose tasks, with controlled state transfer.
  • Resumable Sessions: Auto-saves REPL variables and code, allowing follow-up queries to reuse previous work without re-exploring.
  • Structured I/O: Supports Pydantic models and JSON Schema for both input data and validated output.
  • MCP Integration: Connects to Model Context Protocol servers to expose external tools and resources directly in the REPL.
  • Flexible Backends: Compatible with OpenAI, Vertex AI, Anthropic, and local ACP coding agents.

Related

  • Project
    grishahq/recursive-llm

    A Python implementation of Recursive Language Models that allows LLMs to analyze large contexts by interacting with them via a restricted Python REPL instead of placing the full text in the prompt.

  • Project
    alexzhang13/rlm

    An inference engine and training environment for Recursive Language Models (RLMs) that enable LLMs to handle near-infinite context by programmatically decomposing tasks via recursive sub-calls in a REPL environment.

  • Project
    PrimeIntellect-ai/prime-agent

    An open-source coding and research agent that uses a persistent Python REPL and a self-improving harness to handle long-running, autonomous tasks.

  • Project
    microsoft/LMOps

    A research initiative focused on fundamental technologies to improve LLM prompting, inference speed, and context handling for building AI products.

  • Project
    Trampoline-AI/predict-rlm

    A runtime for Recursive Language Models that allows LLMs to solve complex, adaptive workflows by writing and executing Python code in a sandboxed REPL.