STAGE 0 · 3 NODES

Data the model makes for itself

Where do millions of training examples come from? Increasingly, the model writes them itself — inventing harder questions, working out full solutions, and keeping only the answers that pass a check.

THE 3 STEPS

01Evolve the instruction 02Write the reasoning 03Verify & keep

NODE 01 / 03

Make the question harder

Fine-tuning needs millions of instruction→answer pairs, and humans can't write them all. So a strong model rewrites a plain seed prompt into harder, richer variants — a trick called Evol-Instruct.

Start from a seed prompt — a basic task, the kind that's easy to collect but too easy to teach much.
An operator rewrites it — add constraints, go deeper, make it concrete, or demand step-by-step work.
One seed becomes many — each rewrite is a new, harder example the model then answers itself.
Harder questions teach more than a pile of easy ones — this is where generated data beats plain scraping.

SEED → EVOLVED VARIANTS

Pick an operator — the seed splits into a harder version you can read below.

Add constraints → “Reverse a string in O(n) time and in place, with no built-in reverse and full Unicode support.” More rules, more skill tested.

NODE 02 / 03

Write out the reasoning, not just the answer

A right answer isn't enough to train on — the model should learn how to get there. So the generator writes the full step-by-step solution, and that whole trace becomes the training target.

Answer-only teaches the what, not the how. The model just learns to emit a result it can't reliably reproduce.
A reasoning trace shows every step — the intermediate work that leads to the answer.
Training on the trace teaches the procedure, so the model can redo it on problems it hasn't seen.
Scroll to build the trace one line at a time, then watch it become a single training example.

PROMPT → REASONING TRACE → ANSWER

1 / 5

Answer-only: the model sees just the question and the final answer — it never learns how to reach it.

Step through — each move reveals one line of the worked solution.

NODE 03 / 03

Generate a lot, keep only what passes

Generation is cheap and often wrong, so the model makes many attempts and a verifier throws out the bad ones. Only the samples that pass become training data — this filtering is called rejection sampling.

Every candidate runs a gauntlet of automatic checks before it's allowed into the dataset.
Format check — is the output shaped right: valid JSON, all the required fields, not cut off?
Unit tests / execution — for code and math, actually run it: does it produce the correct result?
LLM judge — a strong model scores quality and cuts the low ones. Only the survivors are kept.

REJECTION-SAMPLING FUNNEL

Toggle the three verifiers — watch the funnel narrow as each one rejects more.

All three verifiers are on — the realistic setup. Only 5 of 12 candidates survive. Turn one off to let more junk through.

EXPLAIN IT BACK

If a model generates its own training data, how does that make it any smarter than it already is — isn't it just echoing itself?

NEXT: REWARD MODELING

A verifier gives a clean yes/no, but most answers aren't right-or-wrong — one is just better than another. Next, the reward model: how a network learns human preference from A-vs-B choices, so it can score answers no rule could check.

← Back: supervised fine-tuning Continue to reward modeling →

Language: English