xavier-ramirez.com
STAGE 0 · HOW MODELS LEARN · 3 NODES

How models learn

Training sounds mysterious, but it's one tiny loop run at absurd scale: guess the next token, measure how wrong the guess was, nudge the weights to be a little less wrong. Play with each part of the loop below.

THE LOOP
01Guess → loss02Nudge the weights03Repeat at scale
NODE 01 / 03

Guess the next token, measure the miss

A language model never really 'knows' the next word — it hands every candidate a probability. Training compares that guess to the token that actually came next. (A token is a word or word-piece.)

  • The model outputs a probability for every possible next token — below are its guesses for The cat sat on the ___.
  • Loss is just surprise. Take the probability it gave the real next word and compute -log(p): low if it was confident and right, high if it got blindsided.
  • That one number is the whole training signal — called cross-entropy, a single score for how wrong each guess was.
  • Confident and right → near-zero loss. Confident and wrong → huge loss.
NEXT-TOKEN GUESS · PICK THE REAL WORD

Pick the real next word — the rarer your pick, the more 'surprised' the model is, and the higher the loss.

The cat sat on the ___

The model basically expected this. Almost no surprise, so the loss is tiny.

NODE 02 / 03

Nudge the weights downhill

One number told us how wrong the guess was. Now we change the model to be a little less wrong — by rolling its weights downhill on the loss.

  • Picture the loss as a valley and one weight as your left-right position. Lower is better.
  • The gradient is just the slope under your feet. It points uphill, so you step the opposite way.
  • Each step: weight ← weight − rate × slope. The learning rate is how big a step you take.
  • Too small and you crawl; too big and you overshoot the valley and bounce out.
GRADIENT DESCENT · ONE WEIGHT

Press step to roll downhill. Nudge the learning rate — too big and the ball leaps past the bottom.

A good rate: each step moves smoothly toward the lowest loss.

NODE 03 / 03

Repeat a few billion times

One guess barely moves the model. Do it across trillions of tokens and the loss slides down a long, curved hill — fast at first, then a slow crawl.

  • Every step is the same loop: guess a token, measure the loss, nudge the weights.
  • Loss falls fast early, then slowly. The easy patterns — spelling, grammar — are learned first; the subtle ones take far more text.
  • More text seen → lower loss. This steady curve is why runs last weeks over trillions of tokens.
  • It never reaches zero — language is genuinely unpredictable, so some surprise always remains.
GOOD TO KNOW · THE STARTING LOSS
A fresh model is just a coin-flip
Before training, every weight is random, so the model spreads its bet evenly over the whole vocabulary — the loss sits near log(vocab), about 10.8. It hasn't learned that the is common or that qx never appears. Every point the curve drops is a pattern squeezed out of prediction errors, never a rule written by hand.
TRAINING LOSS · OVER THE WHOLE RUN

Drag to train — watch the loss fall as the model reads more text.

Early on, the model is grabbing the easy wins — letters, common words — so the loss drops steeply.

EXPLAIN IT BACK
In one sentence, what is 'training' a language model?
NEXT: BASE vs ASSISTANT

This loop gives you a base model — a raw next-token predictor that will happily continue any text. Next: how that raw predictor is turned into a helpful assistant that answers you instead of just autocompleting.

← Autoregressive loopContinue to base vs assistant →
Language: English