huggingface/search-and-learn

Search and Learn – Scaling Test‑Time Compute for Open LLMs

What it is

• A small, open‑source toolkit from Hugging Face that provides recipes and scripts for making large language models “think longer” at inference time. It implements a few search‑based inference strategies (Best‑of‑N, beam search, Diverse Verifier Tree Search) that let you allocate more compute to hard problems such as math or code generation.

Why it matters

• Training ever larger models is becoming prohibitively expensive. By increasing the amount of compute used during inference, you can get better results from a fixed‑size model, similar to what OpenAI’s o1 model demonstrates.

Core pieces

Piece	Role
scripts/	Command‑line utilities that launch the scaling‑compute pipelines (e.g., launching jobs on a cluster, handling model loading, running the search algorithms).
recipes/	YAML configuration files that describe a single inference run: which model, which search algorithm, hyper‑parameters (beam width, number of samples, verifier model, etc.), and any Accelerate/Slurm settings.
src/	Minimal Python library that implements the search algorithms and the interface to “process reward models” (verifiers) used to score intermediate reasoning steps.
tests/	Unit tests ensuring the search logic works as expected.

Getting started

1. Create a fresh environment (Python 3.11 recommended) and install the package in editable mode:

   conda create -n sal python=3.11 && conda activate sal
   pip install -e '.[dev]'
   

2. Log in to Hugging Face so the scripts can pull models and push results:

   huggingface-cli login
   

3. Install Git‑LFS (required for large model files).
4. Pick a recipe from recipes/ (e.g., best_of_n.yaml) and run the associated script as described in recipes/README.md.

Typical workflow

1. Choose a model (e.g., meta-llama/Meta-Llama-3-8B).
2. Select a verifier (a smaller model or a fine‑tuned reward model that can judge each reasoning step).
3. Edit a YAML recipe to set the compute budget (e.g., num_samples: 32, beam_width: 5).
4. Launch the script; it will generate multiple candidate continuations, score them with the verifier, and return the highest‑scoring answer.

Who might use it

• Researchers exploring test‑time compute scaling or comparing search strategies.
• Practitioners who need higher‑quality answers from a fixed model without retraining.
• Anyone wanting to reproduce the results from the accompanying blog post and paper.

Limitations

• Only three search algorithms are currently implemented; more exotic methods would need to be added manually.
• Requires access to a verifier model; training such a process‑reward model is outside the scope of the repo (though the code provides hooks for it).
• The toolkit is geared toward batch/cluster execution (Accelerate/Slurm) and may be overkill for tiny, single‑GPU experiments.

Citation If you use the code or the ideas, cite the repository’s own citation and the DeepMind paper it builds on (both provided in the README).

---

All information above is taken directly from the repository’s README; no external assumptions have been added.