Trampoline-AI/predict-rlm

📚 What predict‑rlm Does

predict‑rlm is a Python library that gives a large‑language model (LLM) a runtime – a sandboxed REPL where the model can write, run, and iterate on Python code while it works on a task. Instead of cramming everything into a single prompt, you describe:

• Inputs (files, strings, images, etc.)
• Outputs (typed results, generated files)
• Tools / skills the model may call (e.g., PDF readers, spreadsheet libraries, custom CLI tools)
• A high‑level procedure (a DSPy signature that acts as a contract)

At execution time the outer LLM receives the contract, then starts a Python REPL inside a sandbox (JSPI/Deno, Pyodide, or an optional Docker‑based sandbox). It can:

• Inspect files, keep mutable state, and branch/re‑try.
• Call sub‑models with narrow, typed DSPy signatures via predict() – even multimodal calls with images.
• Use async tool calls, manage large contexts without “context rot”, and produce a typed, reproducible output.
• Emit a full trace (code, tool calls, timings, token usage, errors) that can be inspected by humans or fed to optimisation tools like GEPA.

When to reach for it

• Complex document or code‑base investigations (audit, compliance, incident analysis).
• Multi‑file data extraction, redaction, or transformation pipelines.
• Spreadsheet‑heavy financial‑model workflows.
• Anything that needs reasoning, branching, or verification before a final answer.

It is not meant for simple one‑shot chat or deterministic ETL where a single LLM call suffices.

---

🛠️ Core Features (as described in the README)

Feature	What it means
Runtime REPL	Model writes and runs Python inside a sandbox, preserving state across iterations.
Typed contracts	DSPy signatures (or simple question -> answer strings) define input‑output types; sub‑model calls are type‑checked via Pydantic/DSPy.
Multimodal support	Images and rendered document pages can be sent to sub‑models using native provider APIs.
Async tool calling	Concurrent sub‑model or external tool invocations inside the WASM sandbox.
Skills & tools	Bundles of domain‑specific instructions, PyPI packages, and host‑side utilities that the RLM can import and use.
Observability	Every run produces trace.json and evidence.json with code, outputs, timings, token usage, and errors.
Optimization‑ready	Traces can be fed to GEPA to automatically improve RLM strategies from scored examples.
Sandbox back‑ends	Default JSPI/Deno/Pyodide sandbox; optional Docker‑based sbx backend for an explicit Linux Python runner.
Spreadsheet skill	Built‑in skill that loads openpyxl, pandas, and formula‑evaluation helpers for spreadsheet‑centric tasks.
Lifecycle callbacks	Hook into iteration start/end (sync or async) to broadcast progress, log, or integrate with UI pipelines.

---

🚀 Quick‑Start Example (from the README)

import dspy
from predict_rlm import CtxStr, File, PredictRLM

class AnalyzeImages(dspy.Signature):
    """Analyze images and answer the query. Load each image as a base64 data
    URI and use predict() with dspy.Image to extract visual information."""
    images: list[File] = dspy.InputField()
    query: CtxStr = dspy.InputField()
    answer: str = dspy.OutputField()

rlm = PredictRLM(
    AnalyzeImages,
    lm="openai/gpt-5.4",
    sub_lm="openai/gpt-5.1",
)

result = rlm(
    images=[File(path="page.png")],
    query="Extract all visible text, then count each letter A‑Z (case‑insensitive).",
)

print(result.answer)

The outer model (gpt‑5.4) drives the REPL, while a narrower sub‑model (gpt‑5.1) is used for the image‑analysis call.

---

📂 Project Structure (high‑level, inferred from README)

• predict_rlm/ – core library (runtime, sandbox adapters, DSPy integration, trace handling).
• examples/ – end‑to‑end demos: document analysis, redaction, invoice processing, contract comparison. Each example stores per‑run traces under .run/<timestamp>/.
• skills/ – reusable skill bundles (e.g., pdf, spreadsheet, rlm).
• tests/ – extensive unit, integration, and backend‑specific test suites (runtime contracts, GEPA, Codex LM, Docker sandbox).
• docs/ – deeper design docs (architecture, observability, custom path inputs).
• CI / packaging – GitHub Actions, Codecov, PyPI distribution, optional extras (gepa, codex‑lm, sbx).

---

🎯 Who Might Use This?

• AI engineers building complex RAG‑style pipelines where a single LLM call cannot handle branching logic.
• Data‑ops / compliance teams that need auditable, reproducible document‑processing workflows.
• Researchers exploring the Recursive Language Model paradigm (the paper the project cites).
• Product teams that want to expose a “skill” (e.g., spreadsheet update) to coding agents like Claude Code, Cursor, or Codex.

---

📦 Installation & Extras

uv add predict-rlm                 # core package
uv add "predict-rlm[gepa]"        # GEPA optimisation support
uv add "predict-rlm[codex-lm]"   # Codex‑backed LM and CLI
uv add "predict-rlm[sbx]"        # Docker sandbox backend

The library works with the Codex LM backend (gpt‑5.6‑terra, etc.) but can also be pointed at any OpenAI‑compatible model via the lm/sub_lm arguments.

---

🧩 Extensibility

• Add new skills – bundle a PyPI package, a custom tool, or a prompt template and list it in skills= when constructing PredictRLM.
• Custom sandbox – implement the backend contract (see ARCHITECTURE.md) and plug it via sandbox_backend=.
• Callbacks – subclass BaseCallback and implement on_rlm_iteration_start / on_rlm_iteration_end for UI updates, logging, or external observability pipelines.

---

📊 Observability & Debugging

• Human‑readable colored trace printed to stderr (disable with verbose=False).
• Exportable JSON: prediction.trace.to_exportable_json("trace.json") and prediction.evidence.to_exportable_json("evidence.json").
• Live event sinks – callbacks receive the same structured data in real time.

---

✅ Bottom Line

predict‑rlm is a genuine, actively maintained open‑source framework that turns an LLM into a programmable, stateful runtime. It targets the emerging “recursive language model” workflow where the model iteratively writes and runs code, calls narrower sub‑models, and produces a fully traceable, typed result. The README provides demos, a solid test suite, optional Docker sandboxing, and integration points for optimisation tools, making it a solid choice for anyone needing sophisticated, auditable LLM‑driven pipelines.