Beating GPT-5.6 Sol on Retrieval with Castform and Neon

Open-Weights Models Can Outperform Frontier Models in Retrieval

Post-trained open-weights models can match or exceed the performance of frontier models like GPT-5.6 Sol on specific retrieval tasks while reducing costs by up to 100x. While general-purpose frontier models are powerful, they are often prohibitively expensive and slow for agentic retrieval workflows—where a model must plan and search multiple times in a loop to solve a problem. By using Reinforcement Learning (RL) post-training, developers can specialize small models to handle these specific search and retrieval patterns more efficiently.

The Shift from Traditional RAG to Agentic Retrieval

Retrieval has evolved from one-shot embedding searches to multi-hop agentic workflows. In traditional Retrieval-Augmented Generation (RAG), a system performs a single similarity search to provide context to an LLM. In contrast, agentic retrieval involves a model that decomposes complex problems into smaller tasks, issuing multiple search queries in a loop until it finds the necessary information.

This shift increases the burden on the model in two key areas:

Context: The ability to provide tools that find the correct data.
Model: The ability of the model to decide what to search for and how to iterate.

For frontier models, this iterative process is costly. A typical multi-turn search request using GPT-5.6 Sol can take over 10 seconds and cost approximately $0.03 per request, which is unsustainable for many production scales.

How Castform and Neon Enable RL Post-Training

Castform and Neon provide an integrated pipeline to turn raw corporate data into a specialized retrieval model without requiring deep expertise in GPU internals or machine learning.

The Training Pipeline

Castform manages the RL loop, while Neon (via Lakebase Search) provides the data infrastructure:

Stage	Role of Neon + Lakebase Search
Corpus Storage	Raw documents are stored in Postgres on Neon.
Synthetic Data Generation	Castform uses `lakebase_text` and `lakebase_vector` to generate training tasks.
RL Training	Every rollout's search tool call utilizes Lakebase Search.
Production Inference	The final model uses the same search tool during live inference.

Turning Data into Tasks

Most enterprises lack clean training datasets. Castform solves this by synthetically generating question-answer pairs from existing proprietary data (e.g., internal wikis, support articles, and product records). For example, a policy document stating "Train rides must be standard cabin class with 14 day booking lead time" is converted into a specific question about booking rules and a corresponding ground-truth answer.

The Reward Function

RL post-training relies on a reward function to guide the model. In retrieval tasks, the reward is calculated based on three factors:

Retrieval: Did the model find the correct data chunks?
Citation: Did it cite the correct sources?
Correctness: Did it provide the accurate final answer?

Infrastructure for Stateful Agents

Training agentic models creates highly bursty workloads, as thousands of parallel rollouts may make dozens of search calls simultaneously. Neon's dynamic compute scaling absorbs these spikes without requiring constant maximum-capacity provisioning.

Furthermore, Neon's branching and time-travel capabilities allow for the training of stateful agents. By creating isolated database branches for each rollout, developers can ensure that one agent's actions do not interfere with others or affect production data, providing a safe environment to reset and inspect agent states.

Community Insights and Technical Counterpoints

Industry practitioners on Hacker News highlighted several critical considerations regarding specialized retrieval models:

"It makes more sense for retrieval, reranking, reasoning, and generation to each have their own optimized model if the routing cost is negligible."

Key Technical Challenges

Data Quality: Some users questioned how the system handles outdated or misleading information within the corpus, as the reward function is derived from the corpus itself.
Retrieval Depth: Concerns were raised about the "needle in a haystack" problem—specifically, the ability of models to find paired needles where one piece of information is required to unlock the discovery of another.
Chunking Strategies: Some argued that the fundamental flaw in RAG is "blind chunking," suggesting that richer parent-child segment models are more effective than simple embedding searches.
Privacy: For sensitive data, the requirement to upload information to a cloud provider remains a significant barrier, with a demand for open-source stacks that can run on rented GPUs.