Kapso: Leeroo's Open-Source Engine for Iterative Learning from Enterprise Data

Leeroo has open-sourced Kapso, an engine that runs experiments against enterprise data, deploys the best solution to a target stack, and updates a git-backed bank of lessons from every run. The project matters because it treats enterprise data work as an environment-specific learning loop rather than a one-shot RAG or upfront ontology exercise.

What Kapso does

Kapso is designed for iterative data and AI work: investigate a system, form hypotheses, run experiments, verify results, and carry the learning into the next experiment. Leeroo argues that this bottom-up process is necessary in large, messy enterprise data environments, where consultants gathering context in meetings or teams defining a business ontology upfront often cannot execute within quarters or years.

The authors describe one enterprise project that moved a company off an old data estate onto a lakehouse with more than 30 source-system families, thousands of tables and transformations, and hundreds of BI reports. No individual or AI agent knew how the estate was connected and operated. The only way to learn it was to investigate repositories, dbt models, documentation, and the data itself, propose hypotheses, run experiments, and learn from the results.

Kapso is intended to automate that loop. A user gives Kapso an objective and success criteria, and the system works toward an optimum solution that can be deployed on the user's stack.

The lesson bank and experiment loop

Kapso's core mechanism is a bank of lessons induced or deduced from past experiments. Each lesson carries a credence score, conditions describing where it applies, and the evidence behind it.

For each new experiment, an ideation module proposes ideas using earlier lessons, re-ranks them by expected return against the budget each would use, and sends the top-ranked ideas to an implementor. Because the bank keeps learning from past experiments in that environment, Kapso's proposed ideas are intended to be better suited to that environment than generic hypotheses from an LLM or RAG.

After each run, a learn() step updates the lesson bank's scores and boundaries using evidence from that run. Changes to the bank can also be A/B tested by running otherwise identical builds with the previous and updated banks.

Verification and evolving success criteria

Kapso verifies experiment results through a verifier module that no experiment can access or cheat. The system also allows the success criteria and evaluation tests to evolve: experimentation modules can propose updates to metrics and criteria, and the verifier approves or rejects those updates.

This matters because, as Leeroo notes, success criteria and evaluation tests sometimes need to change as the system learns more about the environment.

Git-backed experience, CLI, and harnesses

The lesson bank is built on git. Users can share experience with others, list available banks, and update them through CLI commands:

  • kapso experience share
  • kapso experience list
  • kapso experience update

Kapso is model-agnostic and coding-agent-agnostic. Every domain has a harness to start from. Users can write their own domain harness with /kapso:harness in the plugin or through the CLI command kapso harness new <name>, then compound learning over that harness by running experiments.

Deployment to a target stack

Kapso deploys a solution with kapso.deploy(solution, target_stack). The deployment workflow runs a shorter experimentation loop to adapt the best solution to the target stack. Leeroo names Docker, Modal, BentoCloud, and LangGraph as examples of target stacks.

Benchmark results reported by Leeroo

Leeroo reports results on public benchmarks and live competitions. These are the authors' reported results, not independently verified in the source material.

RelBench

Kapso placed first on RelBench, the benchmark associated with Kumo AI, now Nvidia, and Stanford. Leeroo says this shows that keeping learning at the harness level can beat foundational table models, and that updating a bank of lessons costs far less than retraining a tabular foundation model.

MLE-bench

Kapso placed first on OpenAI's MLE-bench at submission and remains among the top open-source systems, according to Leeroo.

ALE-Bench and the International Olympiad in AI 2026

Leeroo says Kapso was ahead of the best agent reported by the ALE-Bench authors at Sakana AI. It also ran autonomously at the live International Olympiad in AI 2026, where Leeroo reports it placed above all 471 human contestants and in the top three AI systems.

Leeroopedia public lesson bank

Leeroo maintains a public bank for ML and data called Leeroopedia. It contains around 100,000 pages extracted by experimenting on papers and repositories. Leeroopedia receives about 6 million requests per month, about 28% of which come from AI agents.

Community feedback

Independent hyperparameter-tuning result on RelArena

A commenter from RelFlow, an open-source model factory, reported using Kapso to help harness a hyperparameter-optimization loop on RelArena. RelArena is described as similar to RelBench but maintained and regulated by Prior Labs to isolate data leakage that can taint modern benchmarks.

Over at https://relflow.ai (an OSS model factory) used Kapso to help harness an HPO loop on RelArena. RelArena is like RelBench, but is maintained and regulated by Prior Labs to isolate data leakage that can taint modern benchmarks. Kapso was able to efficiently calibrate Relflow's hyperparameters on 21 structured data modeling tasks, and we were able to outperform TabPFN on the majority of the larger data modeling tasks despite not using any ICL, and using ~1% of the number of params of TabPFN.

Naming collision with a WhatsApp integration service

A commenter noted that a fairly popular service already uses the name Kapso for integrating WhatsApp into other products. The two projects are unrelated.

FYI, you may want to reconsider the naming as there's already a fairly popular service named Kapso [1] for integrating WhatsApp into other products (not related btw). [1] https://kapso.com/

What Kapso is trying to change

Kapso's central claim is that enterprise data work improves through environment-specific experimentation and compounding lessons, not through generic documentation retrieval or a fixed upfront model of the business. The lesson bank, verifier, deployment loop, and git-backed sharing mechanism are the concrete pieces Leeroo has open-sourced to test that claim.

Sources