xavier-ramirez.com
STAGE 0 · 3 NODES

Reasoning tokens

A "reasoning" model doesn't have a second brain. It just writes a hidden scratchpad before its answer — bracketed by two special tokens — spending extra compute to think out loud where you can't see it. Under the hood it's the same next-token loop.

THE 3 STEPS
01The think block02Why it helps03Still autoregression
NODE 01 / 03

The think block

Reasoning is switched on by two ordinary tokens. <think> opens a scratchpad region; </think> closes it. Everything between them is the model's private working — generated first, then hidden from you.

  • <think> and </think> are special tokens — single entries in the vocabulary, each with its own id, exactly like a word-piece. They just delimit a region.
  • The scratchpad comes first, the answer comes last. The model fills the region with working, emits </think>, then writes the reply the user actually reads.
  • A parser hides the scratchpad. The app shows you everything after </think> and drops the rest — so the reasoning is there, but off-screen.
  • Turn reasoning off and the model blurts. On this classic riddle the fast, intuitive answer is wrong; the scratchpad is what gets it right.
GENERATED STREAM · THINK BLOCK

Toggle Reasoning, then flip the view — the scratchpad is generated either way, the user just never sees it.

Reasoning ON — the model opens a `<think>` block, works through the algebra, closes it, then answers. The intuitive trap is avoided.

NODE 02 / 03

Why more tokens buy better answers

Every token the model generates is one forward pass — one full sweep through the network. A blurted answer gets one sweep to decide. Letting it write a scratchpad first buys many more sweeps before it commits.

  • Compute is spent per token, not per question. More generated tokens = more forward passes = more thinking before the answer.
  • This is "test-time compute": you pay for quality at answer-time, not by making the model bigger.
  • Give it a budget of reasoning steps below. Each step is a few more scratchpad tokens — more compute, more of the derivation done.
  • There's a threshold, then diminishing returns. Too few steps and it can't finish the algebra; past enough, extra tokens mostly waste time.

Illustrative: real accuracy rises with more thinking, then plateaus.

THINKING BUDGET → COMPUTE → ANSWER

Drag the budget from 0 — the answer stays wrong until the model has enough steps to finish the algebra.

Budget 0 = blurt: one forward pass, the intuitive wrong answer. Each step adds scratchpad tokens — more passes, more of the working done.

NODE 03 / 03

It's still just the loop

Nothing new happens mechanically. <think> is generated the same way as any word: predict the next token, append it, feed the whole context back in — the loop from the autoregressive lesson. The tags are just tokens the model learned to emit.

  • Same append-and-feed-back loop. Each new token — delimiters included — joins the context and the model predicts again.
  • Special tokens ride the same tape. <think>, </think> and the end token <eos> are vocabulary entries with ids, sitting between ordinary tokens.
  • The model isn't in a "reasoning mode." It generated a <think> token because training taught it to; the only extra machinery is the parser that hides the region.
  • Step through the stream — watch each token get appended and fed back, delimiters and answer alike.
ONE TOKEN AT A TIME
1 / 7
The model predicts `<think>` — a special token, id and all — and appends it.

Step through — the dashed tokens are special vocabulary entries, but they're generated by the same loop as the rest.

dashed = special vocabulary token · same tape, its own id
EXPLAIN IT BACK
A reasoning model and a plain model run the exact same next-token loop. So what actually makes the reasoning one "think"?
NEXT: TOOL CALLING

The scratchpad lets a model think — but it still can't do anything outside its own text. Next, tool calling: how the same token stream can emit a structured request the runtime executes — a search, a calculation, an API call — and feed the result back into the loop.

← Back: constrained decodingContinue to tool calling →
Language: English