Notes on how AI systems actually work

Engagement counts what happened; efficacy asks whether it mattered. Section-level randomisation, a power calculation with the classroom clustering priced in, and the single modelling assumption that flips the same children and the same scores from “no detectable effect” to a number somebody would have published as a win.

12 min read

Diff, don’t republish. Re-importing a course strands sixty-three pieces of student work on the first night and a diff strands none — then read the funnel from what a teacher approved to what a child can actually use, and learn which of the losses are the gate working correctly.

11 min read

A guardrail that runs only at generation is a default, not a guardrail. Publishability as a join the editor cannot route around, every teacher edit stored as an observation rather than an overwrite, and a deletion fan-out with one destination no amount of engineering can reach.

12 min read

“Age-appropriate” is four measurements wearing one word. Drive a rubric over real blocks and watch the same one pass in one district and hold in another — then see a same-family judge report 94% while the teachers who actually use the content rewrite six lessons in ten.

12 min read

A unit is a connected component, and alignment claims must be retractable. One well-meaning district mapping welds a whole strand into a single blob that reports 100% coverage — turn the guards on and watch it fall apart into standards, with coverage that finally tells the truth.

11 min read

A standard code to a published assignment, end to end. A request that is a contract rather than a prompt, generation as a queue consumer, and immutable content-addressed blocks — run the same request four ways and watch where the model call lives decide whether an outage is lag or a broken first period.

10 min read

The chain from a curriculum code to a measured mastery gain, in six hops — and why generation stopped being the expensive part. Drive the review queue that decides whether a thousand generated lessons ever reach the handful of people who can catch what is wrong with them.

10 min read

Attribution divides credit; incrementality asks whether it mattered. A deterministic holdout the sync engine enforces, a power calculation done before the test, and an interactive interval that shows why a small true lift is invisible at any audience a brand actually has.

11 min read

Send what changed, not what is. Snapshot, diff, batch to the destination's declared limits, dead-letter the rejects — then read the match-rate funnel and learn which losses are defects and which are the consent join working exactly as designed.

11 min read

Consent as a join key the audience compiler cannot skip, brand isolation with a row policy underneath it, and a deletion that fans out to every destination you ever synced to. Delete one line from a WHERE clause and watch the audience quietly grow.

11 min read

Traits as a materialised query over the event log, one feature definition compiled into two paths, and the point-in-time leak that makes a model look brilliant offline and do nothing in production. Slide the feature cutoff and watch the offline score rise as the model gets worse.

11 min read

A profile is a connected component, not a row. Links carry provenance so a merge can be undone — and one shared store terminal welds four households into a single customer whose reported lifetime value is the sum of all of them. Turn the guards on and watch it fall apart into households.

10 min read

Anonymous click to synced audience, end to end. A first-party server-side collector, an event envelope you can trust, and five stages on one streaming backbone — then follow a single device through every hop and watch it stop dead when the identify event has not arrived yet.

9 min read

The chain from an anonymous session to a measured incremental dollar, in six hops — and the one link everything downstream is a bet on. Includes the cookie demonstration that explains why the collector moved server-side, and a map of the six posts that build the platform.

9 min read

The groundedness gate between a demo and production. A threshold decides ship-or-abstain, a CI eval blocks a bad release, and prompt-injection and PII screens catch the rest — the line between a RAG demo and a system you can run in a regulated shop.

10 min read

Every answer cites the file it came from — or it abstains. Trace each [n] marker back to a source chunk and its system of record, and watch the groundedness score fall below threshold and force an abstention instead of a confident guess.

9 min read

Filter at retrieval, not after. The permission check belongs inside the retrieval query, built from the caller's signed token — flip a switch and watch a restricted underwriting memo and another tenant's data leak into the answer when it isn't.

10 min read

Cheap dense retrieval casts a wide net; a reranker sharpens precision so the causal passage lands in the top few that fit the context budget. Watch a passage ranked #9 by vector search become #1 after rerank, and see why exact codes still want hybrid search.

9 min read

Chunking is a retrieval decision, not preprocessing. How you cut a claim or policy document decides whether the answer-bearing detail survives in one passage or gets orphaned across two — drive a live chunk-size and overlap splitter and watch the answer break.

9 min read

Rebuild Google's AI Overview over your own private data. The whole pipeline, thin but real: a write path that ingests change data from an S3 data lake (trigger on the commit, evict on tombstones) and a six-node LangGraph read path that answers with citations. Clone it and run it, zero cloud.

8 min read

Type a question, get a cited summary. Open the black box: the query fan-out, the live retrieval, and an answer where every sentence traces back to a source you can see — with a working demo you can drive.

9 min read

Spend more compute at inference to get better answers. Play with three live diagrams — Best-of-N sampling, tree search (MCTS), and the compute-vs-quality trade-off — to see how a…

10 min read

A small draft model guesses several next tokens; the big model verifies them all in one pass and accepts the correct prefix.

11 min read

Serving a model has two phases: prefill reads the whole prompt in one fast parallel pass, then decode writes the answer one token at a time.

12 min read

A serving engine manages the KV cache like an OS manages memory. Play with three live diagrams — fragmentation, PagedAttention's fixed pages, and RadixAttention's shared prefixes — to…

8 min read

Why generation is limited by memory, not compute. Three live diagrams: the rising staircase of wasted work without a cache, the collapse to a flat line with it, and the KV cache eating…

12 min read

One token isn't a sentence. Play the loop that writes a whole paragraph: the model predicts a token, appends it, feeds it back, and repeats until an end-of-sequence token — the reason…

6 min read

Guarantee valid JSON, every time. Three live diagrams show why free sampling breaks, how logit masking forces invalid tokens to zero probability, and how a grammar decides exactly…

11 min read

You have a probability for every next token — now how do you pick? Play with three live bar charts: temperature flattens or sharpens the odds, top-k / top-p chop off the unlikely tail,…

9 min read

A transformer's last step turns one hidden state into a single next token. Scroll through three live diagrams — the unembedding that scores every token, the softmax that turns scores…

10 min read

A model can't check the weather or run your database — it only predicts text. Step through the tool-calling loop with three live diagrams: the model emits a structured call, the…

9 min read

A model has no memory. Scroll through three live diagrams — a growing stack of role blocks, a tool result folding back in, and a pre-filled <think> scratchpad — to see how long chats,…

8 min read

A reasoning model has no second brain — it writes a hidden scratchpad before its answer, bracketed by <think> tokens.

10 min read

For math and code you can skip the learned reward model and check the answer for real. Scroll through three live diagrams — a verifier scoring candidate answers, outcome vs process…

7 min read

GRPO removes PPO's memory-heavy critic. Scroll three live diagrams — the PPO-vs-GRPO memory stacks, one prompt fanning into a group of answers, and each answer scored against the group…

10 min read

PPO is the online RL loop that improves a model from a reward model. Scroll through three live diagrams — the actor–critic cast, the clipped step, and the KL leash — to see how it gets…

12 min read

DPO skips the separate reward model and the RL loop. It takes the same chosen-vs-rejected pairs and nudges the model directly to prefer the chosen answer over the rejected one — pick a…

10 min read

You can't have a human grade every answer. Scroll through three live diagrams — labelling preference pairs, training a scalar reward, and ranking fresh answers — to see how a model…

8 min read

A freshly pretrained base model only continues text. Play with three live diagrams — the same prompt answered two ways, the instruction → response pairs that teach it, and what…

9 min read

An instruction-following assistant can still be unhelpful or unsafe. Play with three live diagrams — pick the better of two answers, watch that preference become a reward score, and…

11 min read

Store each weight in 4, 3 or 2 bits instead of 16 and a model shrinks from a data-centre GPU to a laptop.

9 min read

How a small model learns from a giant. Scroll through six live diagrams — the memory bottleneck, soft-target logits, behavioral cloning, on- vs off-policy distillation, edge deployment…

8 min read

LoRA freezes a giant model and trains a tiny low-rank adapter instead — so a huge model fine-tunes on a single GPU.

11 min read

How a base model becomes an assistant: train it on instruction→answer pairs and grade it only on the answer.

8 min read

How clean text becomes GPU-ready batches: pre-tokenize the corpus once, stitch documents into one stream with <eos> markers, and pack the stream into fixed-length blocks with zero…

6 min read

A frontier model is too big for one GPU. Scroll through three live diagrams — the data-parallel GPU grid, tensor-vs-pipeline model splitting, and the ZeRO/FSDP memory bar — to see how…

11 min read

Training runs in low-precision numbers for speed and memory. Scroll through three live diagrams — the bit layouts of FP32/BF16/FP8, gradient survival under loss scaling, and a one-GPU…

10 min read

AdamW stores two extra numbers per weight. Play with three live diagrams — the per-weight memory tax, optimizer-state bytes for Lion, Adafactor and Muon, and loss-vs-steps convergence…

7 min read

Backprop gives the gradient; the optimizer decides the step. Scroll through three live diagrams — SGD vs Adam on a loss surface, per-weight step sizes, and the warmup-then-decay…

8 min read

How one wrong guess tells millions of weights which way to move. Scroll through three live diagrams — blame flowing backward through a tiny network, the gradient arrow on a loss bowl,…

8 min read

Training is one tiny loop at absurd scale: guess the next token, measure how wrong it was (loss), nudge the weights, repeat.

10 min read

A model trained on 4k tokens can read 128k without retraining. Scroll through three live diagrams — the RoPE dial that breaks past its training length, the frequency rescale that fixes…

9 min read

Instead of one big feed-forward per token, a router sends each token to a few specialist experts.

7 min read

A transformer is one small block stacked over and over. Play with three live diagrams — attention links over a token row, the parts of a block (mix, think, shortcut), and a stack of…

9 min read

After attention, each token runs through a small 2-layer network that expands the vector, fires a gate, and compresses it back — where most of a model's facts are stored.

7 min read

Attention's N×N score grid grows with the square of the sequence. Play with three live diagrams — the memory wall, tiling the grid through fast on-chip memory, and GQA/MQA key-value…

7 min read

The transformer block showed each token looks at the others; this is the machine that does it.

9 min read

RoPE encodes a token's position by rotating its query and key vectors. Play with three live dials to see why only the gap between two positions survives — and why rotating beats adding…

6 min read

Attention reads every token at once, so on its own it can't tell word order apart — \

9 min read

A token ID is just a number. Play with three live diagrams — the matrix lookup that turns an ID into a vector, the 2D meaning map where similar words become neighbours, and the…

7 min read

How does a whole chat become tokens? Scroll through three live diagrams — the conversation array, the chat template that folds it into one string, and a reserved control token as a…

9 min read

Models don't read letters or words — they read tokens from a fixed dictionary. Three live diagrams: letters vs words vs sub-words, a Byte-Pair Encoding trainer that builds the…

7 min read

Part 2 of the data pipeline: twelve live, interactive deep-dives under the same four stations — the robots.txt gate, retrain-vs-retrieve, prompt injection, connected-code cleanup,…

6 min read

Strong models write their own training data. Scroll through three live diagrams — evolving a seed prompt into harder variants, building a step-by-step reasoning trace, and a…

6 min read

A clean corpus isn't a training set yet. Scroll through six live diagrams — the domain mixer, temperature sampling, the epoch cliff, synthetic textbook generation, the 3-gate synthetic…

8 min read

A well-written page can still be hateful, explicit, or dangerous. Scroll through three live diagrams — the harm categories, the classify-then-cut gate, and the strictness tradeoff — to…

10 min read

Before training, personal data — emails, phones, IDs, names — and copyrighted or sealed text are detected and stripped.

11 min read

The open web repeats itself constantly. Scroll through six live diagrams — the memorization gate, exact hash + suffix-array trimming, a real MinHash estimator, the LSH S-curve,…

9 min read

Hand-written rules only catch obvious junk. Train a small classifier on good-vs-bad text, score every page 0–1, and keep only the high scorers — the FineWeb-Edu idea, in three live…

9 min read

Most of the crawled web is junk. Scroll through three live diagrams — how little of a page is prose, the four cheap heuristic rules that catch menus, spam and gibberish, and the funnel…

12 min read

A crawled page is mostly boilerplate. Scroll through three live diagrams — strip a page to the article, turn hard formats (PDFs, scans, tables) into clean markdown, and tag the…

9 min read

Web text is thin on reasoning. Scroll through six live diagrams — the data-recipe mixer, AST repo packing, copyleft scrubbing, two-stage PDF/OCR routing, temporal decay of Q&A, and…

8 min read

Nobody downloads the internet. Scroll through six live diagrams — the authority field, the byte treemap, the WARC→.bin flow, the robots.txt gate, retrain-vs-retrieve, and prompt…

9 min read

From the whole internet to a training recipe. Walk the four stations that turn the open web into a small, clean, carefully-blended corpus — gather, extract, deduplicate, mix — across…

10 min read

The six steps that turn raw text into a model you can talk to: gather data, clean & tokenize, pretrain, fine-tune & align, evaluate, and host. A clickable pipeline.

10 min read

A model mirrors its corpus's skews — bias is the data's statistics learned faithfully. See who's in the data vs. the world, and why every cleaning choice is a value choice.

7 min read

A model's knowledge is just the text it was fed. Clean a real dataset stage by stage, watch most of it get thrown away, and see the token tape a model trains on.

7 min read

A fixed embedding can't tell riverside “bank” from money “bank.” Attention is the fix: each token looks at the others and blends in what's relevant.

7 min read

Every token becomes a vector — a point on a map where nearby means similar in meaning. Explore the map, watch it settle out of pretraining, and try king − man + woman ≈ queen.

12 min read

A model never sees your words, it sees tokens. A live tokenizer lesson on cost, context limits, and why models miscount letters.

11 min read

A neural network is a mountain of matrix multiplies — that's why AI runs on GPUs, not CPUs. Race the two machines, tour a rack of GPUs, and split a model across the fleet.

6 min read

A neural network is a function you fit to data — stacked multiply-and-adds with a nonlinearity. Wire up a real tiny network by hand and watch the decision boundary bend.

8 min read