Poolside’s Model Factory, Laguna S, and Open Models: Eiso Kant on Building Code Models for AGI

Model Factory Enables Rapid Experimentation

Poolside’s Model Factory reduces model development from months to weeks by integrating streaming data, versioned code, and automated agents. The team moved from six‑month model cycles to five‑ and eight‑week launches for models like Laguna XS2 and Laguna S. They run 10,000 to 20,000 experiments per month, trusting the machine enough that changes made a day before can go into the next run. Agents write code, launch training jobs, evaluate results, and modify pipelines, shrinking the human loop. Immutable data and versioned code give perfect reproducibility, allowing any experiment to be rerun exactly.

Laguna S Prioritizes Persistence Over Raw Intelligence

Laguna S achieves strong performance through verification, backtracking, and persistent reasoning rather than scale alone. The model has 118 billion total parameters with 8 billion active, fits on a DGX Spark, and runs at about 30‑40 tokens per second. Peng, a co‑head of applied research, said the gains in Laguna S come "not from more intelligence, but more from different behavior, more verification, less taking things for granted, not declaring victory early, and being way more persistent." The model can solve Erdos 397 independently and handle complex programming tasks without external libraries by persisting through trial and error.

Open Weights and Open Research Differ

Releasing model weights alone does not enable replication; sharing research details is needed for true openness. Giving someone the weights does not allow them to recreate what you are doing, as the underlying research, data mixes, and training pipelines remain hidden. Poolside therefore publishes technical reports that detail how models are built, including data sources, optimizer choices, and training procedures. They view open research as the meaningful contribution that lets others learn from tens of thousands of experiments.

Global Talent Strategy Avoids Bay Area Competition

Poolside built a distributed team outside the Bay Area to access worldwide expertise and avoid talent wars. From day zero the founders decided not to hire any researchers in the Bay Area, instead looking everywhere from middle America to Serbia, Taiwan, and Singapore. They opened offices in Paris and London while keeping a presence in the US, believing that the best innovative ideas are not confined to one region. This global approach created resilience and persistence in the team, with very few people leaving over the years.

Model Building Is Mostly Engineering

Over 90% of work in foundation model development is engineering, not theoretical breakthroughs. Poolside’s early view was that model building is ultimately 90% engineering, with researchers spending most of their time writing code, examining data, and running experiments. The company treats the process as an industrialized system where each component—data pipelines, distributed training, reliability—has its own machinery.

Data and Compute Efficiency Drive Gains

Improving data quality or compute efficiency accounts for most progress in model building. Eiso estimates that 95% of model building can be reduced to just two levers: better data or better compute efficiency. Advances such as low‑precision training (FP8 for Laguna S) and smarter networking squeeze more capability from existing hardware. Better data involves filtering, cleaning, transformations, and creating curricula that teach the model to reason earlier in training.

Streaming Data and Reproducibility Accelerate Research

Feeding data directly into training and maintaining immutable datasets allow rapid, trustworthy experiments. Instead of materializing a full data set before each run, Poolside streams data just‑in‑time, so training can start while the rest of the data is still arriving. The data layer is treated as immutable and code as versioned, letting researchers trace every token back to its exact source and code version. This reproducibility turned earlier YOLO‑style runs into rigorous scientific experiments where each trial is a clean ablation.

Agents Automate the Model Factory

Agents now write code, launch training jobs, evaluate results, and modify pipelines, reducing human loop time. When walking through the office, Eiso sees researchers’ screens filled with agents that are coding, submitting jobs, checking outcomes, and tweaking the training infrastructure. The agents operate on both pre‑ and post‑training pipelines and synthetic data generation, and are beginning to influence architecture decisions. This automation creates early signs of recursive self‑improvement, where the system improves its own training process.

Reinforcement Learning Will Shift Earlier

Eiso expects reinforcement learning to move into pre‑training to teach reasoning sooner. He holds a not‑commonly‑held opinion that RL will move earlier and earlier into pre‑training, rather than staying a separate post‑training phase. Early RL could induce reasoning capabilities while the model is still seeing raw web text, turning next‑token prediction into a thinking process.

Next‑Token Prediction Underutilizes Web Knowledge

Current pre‑training extracts too little from the web; better methods are needed to turn raw text into reasoning. Eiso argues that next‑token prediction pre‑training is not enough and that we are still barely squeezing out of the web what we should be getting. Research is underway to use the web as a way to teach models to think earlier in training, moving beyond pure statistical prediction.

Vision Modality Prioritized, Audio Delayed

Poolside focuses on visual understanding as the next step after language, postponing audio work. The models currently lack vision understanding, but the team considers it super important and has started efforts to build it. Audio is not a near‑term priority because Eiso does not believe it pushes the system closer to AGI, preferring to invest compute in language and vision.

Naming and Ambition Behind Poolside

The name Poolside reminds the founders to never lower their ambitions, inspired by a chance meeting at a conference. During an early conversation with a potential partner, the chief scientist suggested moving to the restaurant poolside, and the name stuck as a reminder to pursue the harder path rather than taking an easy acquisition deal. It symbolizes refusing to round down ambitions when building foundation models.

Fundraising and Investor Skepticism

Poolside raised $500 M while many investors still doubted AGI’s feasibility. During the fundraising round, the majority of investor conversations were still trying to explain that these models were not just stochastic parrots and that they would keep improving. The capital was secured despite skepticism about whether AGI was a real near‑term prospect.

Risks of Over‑Restricting Open Models

Premature restrictions on open model releases could concentrate power and hinder innovation. Eiso prefers a world with 100 foundation model companies over a world with only five, even if Poolside were one of the five. He warns that regulation or safety concerns that lock out open releases could accidentally create an oligopoly of two or three AI companies, similar to how advertising bans on cigarettes entrenched a few tobacco firms.

Hardware Bottlenecks and RL Wall‑Clock Time

Reinforcement learning’s wall‑clock time, limited by batch size, remains a major bottleneck despite hardware advances. Because RL tasks are finite, increasing batch size is difficult, so scaling compute for RL does not follow the same curve as pre‑training. Eiso cites this as one of Poolside’s biggest current bottlenecks and looks forward to techniques that separate prefill and decode across different chips to improve RL efficiency.

Training from Scratch Rather Than Distilling

Poolside trains models from scratch instead of simply distilling larger models to retain full research control. Distilling would hide the lessons learned from the training process, making it harder to improve the model factory. Training from scratch lets the team study stability, data mixes, and optimizer behavior at each order of magnitude, feeding those insights back into the system.

Hiring Across All Model‑Building Functions

The company recruits researchers and engineers for pre‑training, post‑training, architecture, evaluation, and engineering roles. Poolside hires across the full spectrum of model building, seeking high‑agency individuals who can take ownership of experiments and infrastructure. With fewer than 70 researchers and about 35 engineers, each hire has a large impact on the model factory’s speed and capability.