Quality filtering threw out the junk pages. This finer pass reaches inside the pages that stay and strips the private and protected bits — emails, phone numbers, IDs, names, plus copyrighted and sealed text — before a single training step. Here's how the scrubber sees a page, and why the law leaves labs no choice.
Personal data isn't one thing. It splits by how you catch it — some has a fixed shape a pattern can match, the rest hides in prose and needs a model.
PII = personally identifiable information — anything that points back to a real person: their email, phone, ID numbers, name, address.
Structured PII has a rigid shape. Emails, phones, SSNs and cards follow strict formats, so a regular expression — a text pattern — catches them cheaply and exactly.
Unstructured PII is just words. A name or an address has no fixed shape, so it takes a named-entity recognizer (NER): a small model trained to tag people and places.
Method follows shape. Pick the wrong tool and you miss half of it — the whole detection design starts with this split.
PII CATEGORIES · BY DETECTOR
↳ Tap a category — see a real example and how it gets caught.
Email — a rigid shape (name@host.tld). One short pattern catches it with almost no false alarms.
NODE 02 / 03
Detect the spans, then mask them
Now run the detectors over a real support ticket. Each one marks the spans it recognizes; redaction then masks them with a typed placeholder before the text ever reaches training.
Toggle each detector — four patterns (email, phone, ID, card) and one model (names & places).
Detect = mark the spans; redact = replace them with a typed placeholder like [EMAIL], so the sentence keeps its shape but loses the secret.
Turn one off and its data leaks through into the corpus — that's a recall miss, the failure that matters most here.
Placeholders beat black boxes. Swapping in [NAME] instead of deleting lets the model still learn the grammar around the hole.
SUPPORT TICKET · DETECT & MASK
1 / 2
Detectors mark every span they recognize, colored by type.
↳ Switch a detector off and scroll to the mask step — watch that field survive into the training text.
Email — the pattern is basically name@host.tld. Near-unambiguous, so its precision is high.
NODE 03 / 03
Why scrubbing is the law, not a nicety
Models memorize. Feed a secret in often enough and the model can recite it back — which turns a privacy chore into a legal one.
Regurgitation — a model can reproduce rare training strings word for word. Leave an SSN in and someone can prompt it back out.
The law forces the cleanup.GDPR grants a right to erasure; copyright and sealed-record rules bar training on protected text at all.
Copyrighted and sealed blocks get stripped too — the same pass drops paywalled articles, licensed books, and legally sealed records.
Scrubbed data can't leak. If the secret was never in the weights, no prompt can pull it out.
GOOD TO KNOW · THE LAW
GDPR, CCPA, and takedowns
Europe's GDPR and California's CCPA give people a right to have their data deleted — nearly impossible to honor once it's baked into billions of frozen weights. So the cleanup happens before training, and copyright holders can still file DMCA takedowns over memorized text.
PROBE THE MODEL · RAW vs SCRUBBED
PROBE PROMPT
What is Jordan Reyes's SSN?
MODEL OUTPUT · TRAINED ON RAW
It's 402-11-9832.
↳ Pick a probe and flip RAW vs SCRUBBED — the raw model coughs it up.
Personal data — a memorized SSN. The classic privacy leak that scrubbing exists to stop.
EXPLAIN IT BACK
A pattern catches emails perfectly, but a name needs a whole model. Why not just use patterns for everything?
NEXT: BIN-PACKING SEQUENCES
The corpus is now clean and private. Next it has to be packed — thousands of variable-length documents fitted into fixed-size training sequences with as little wasted space as possible, so no GPU cycle trains on padding.