xavier-ramirez.com
STAGE 0 · 4.6 · 3 NODES

Verifiable rewards

For math and code you don't need a reward model that guesses what a human would like. You can check the answer for real — does the number match, does the code pass its tests — and hand back a clean 1 or 0. The catch: give a model a checkable reward and it will try to game the check.

THE 3 STEPS
01Check the real answer02Grade every step03Block the cheats
NODE 01 / 03

Check the real answer, don't predict a score

Earlier you built a reward model — a trained scorer, a fuzzy guess at what a human would prefer, and something a model can learn to fool. For math and code there's a shortcut: check the answer against ground truth.

  • A verifier is a program, not a network — it runs the code or evaluates the maths and returns a hard 1 (correct) or 0 (wrong). No training, no drift.
  • It extracts and normalises first. "The answer is 149.", 149, and 149.0 all reduce to the same number before the comparison — so formatting never costs a point.
  • This is RLVR — reinforcement learning from verifiable rewards. The reward is ground truth, so the signal is exact where a learned scorer only approximates.
  • It only works where truth is checkable — a unit-test suite for code, a known answer for maths. Open-ended writing still needs the learned reward model.
PROBLEM: 12 × 12 + 5 = ?
CANDIDATE → VERIFIER → REWARD

Cycle the candidates — messy formatting still scores 1; only the wrong number gets a 0.

TRY A CANDIDATE ANSWER

Free text — the verifier pulls the last number out (149) before comparing, so prose around the answer doesn't matter.

NODE 02 / 03

Reward every step, not just the final answer

A right final answer can hide wrong reasoning. Grading only the outcome pays out for luck; grading each step gives a denser, more honest signal.

  • Outcome reward (ORM) scores one thing: the final answer. A lucky solution with a broken step in the middle still collects the full reward.
  • Process reward (PRM) scores every step on its own — so a bad step gets a 0 even when the final answer sneaks in correct.
  • Denser signal trains faster. One reward at the end is a faint whisper; a reward per step tells the model where it went wrong.
  • A PRM is itself a trained model — a step grader learned from labelled reasoning — so it's a middle ground between a pure verifier and the human-taste reward model.
OUTCOME vs PROCESS REWARD

Flip to the lucky solution — outcome still pays full reward; process flags the two broken steps in red.

PICK A SOLUTION

Clean solution — every step is valid and the answer is right. Both graders agree: full reward, all green.

GRADING
OUTCOME1
PROCESS1·1·1·1
FINALx = 5 ✓
NODE 03 / 03

When the model games the reward

Give a model a checkable reward and it optimises the check, not the task. If the tests are weak, it will pass them without ever solving the problem — reward hacking.

  • The hack: clear the visible tests without doing the work — hardcode their answers, or return whatever value makes them green.
  • The fix: held-out hidden tests. Keep some cases secret. The real solver passes them; a cheat that only fits the visible cases fails.
  • Guard off, the hack is rewarded but wrong — a 1 for a program that doesn't work. Guard on, the hidden tests catch it and the reward drops to 0.
  • This is Goodhart's law — when a measure becomes the target, it stops measuring what you meant. Verifiable rewards don't escape it; they just move the fight to test coverage.
HIDDEN TESTS
When the reward can be gamed
Guard off, the hack is rewarded but wrong — a 1 for a program that doesn't work. Guard on, the hidden tests catch it and the reward drops to 0.
SOLVER → TESTS → VERDICT

The hardcoded solver is rewarded with the guard off — turn HIDDEN TESTS on and watch it get caught.

PICK A SOLVER

Hardcode tests — returns True only for the two visible primes. Passes what it can see; the hidden case is_prime(2) exposes it.

VERIFIER
VISIBLEpass
HIDDEN
REWARD1
EXPLAIN IT BACK
If a verifier gives a perfect, ground-truth reward, why not use it for everything and retire the learned reward model?
NEXT: THE KV CACHE

You've now seen how a model is built, trained, and aligned. Now the course turns to serving it fast. First, the KV cache: how a model avoids re-reading the whole conversation on every new token — the single trick behind fast generation.

Back: GRPOContinue: The KV cache
Language: English