xavier-ramirez.com
STAGE 0 · TEST-TIME SEARCH · 3 NODES

Test-time search

A trained model gives one answer per prompt — unless you spend more compute at inference. Generate many answers and keep the best, or search a tree of reasoning paths. Same frozen weights, better answers, more compute.

THE 3 STEPS
01Best-of-N02Tree search (MCTS)03The trade-off
NODE 01 / 03

Best-of-N: sample many, keep the best

The simplest way to spend more compute at inference: don't settle for the first answer. Sample N of them, then keep the single best.

  • Sample N answers to the same prompt with a non-zero temperature — the randomness knob from the Sampling strategies lesson — so each one comes out different.
  • Score each with a reward model — a second model trained to grade answers — and keep the highest scorer.
  • Quality climbs with N, fast at first: the best of 12 tries beats the best of 2. But every extra sample is another full generation.
  • It's the baseline for test-time compute — no training, just a scorer and a loop, and often shockingly effective.
BEST-OF-N · N SAMPLES, ONE KEPT

Drag N from 1 to 12 — more tries buy a better best answer, but the average barely moves.

Each extra sample is a full generation. The best score jumps early, then plateaus — classic diminishing returns.

BEST_OF_N
SAMPLES6
BEST SCORE71
MEAN52.0
What N samples bought: how many you generated, the best reward score among them, and the average (which barely improves — you're selecting, not improving each answer).
NODE 02 / 03

Tree search: spend the budget where it pays

Best-of-N generates every candidate in full, then throws most away. Tree search is smarter — it scores partial reasoning and pours compute into the branches that look promising.

  • Reasoning is a tree. Each step forks into a few next steps; a full answer is one path from root to leaf.
  • A value model scores partial paths — a guess at the final quality — so you don't have to finish a branch to judge it.
  • Spend the budget best-first — the heart of MCTS (Monte-Carlo Tree Search): expand the most promising node, score its children, repeat.
  • You reach the best leaf without generating them all — the win over Best-of-N when finishing an answer is expensive.
TREE SEARCH · EXPAND THE PROMISING BRANCH

Drag the budget from 0 to 4 — the search scores every first step, then digs into the best one before the others.

Spend the budget on the most promising branch (B) first, expanding only its children. It already finds a 90.

TREE_SEARCH
EXPANSIONS2
NODES6 / 10
BEST PATH90
The state of the search: expansions spent, tree nodes revealed so far, and the score of the best complete path found.
NODE 03 / 03

The trade-off: compute for quality

More inference compute buys more quality — but not for free, and not forever. Here's the same budget spent two ways.

  • Both curves rise, then flatten. The first few samples help a lot; the twentieth barely moves the score.
  • Tree search gets more quality per unit compute — it prunes dead branches instead of finishing them.
  • But it needs a value model and more plumbing. Best-of-N needs nothing but a scorer and a for-loop.
  • The real trade-off is compute vs latency vs quality — spend it where the answer is worth the wait.
SAME BUDGET, TWO METHODS

Slide the budget up — quality climbs fast, then flattens; tree search stays ahead the whole way.

Illustrative scores — the shape (rise then flatten, tree above Best-of-N) is the real lesson, not the exact numbers.

EXPLAIN IT BACK
You have a fixed inference budget. When is tree search worth the extra machinery over plain Best-of-N?
NEXT: MIXED PRECISION

Searching at inference means running the model many times — so every run has to be cheap. Next, mixed precision: storing and multiplying weights in fewer bits (FP16, BF16, FP8) to make each of those forward passes faster and smaller.

← Back: tool callingContinue: mixed precision →
Language: English