xavier-ramirez.com
← Appropriateness

Your own lesson pipeline · 04 · 29 Aug 2026 · 12 min read

Guardrails, tenancy & deletion

A guardrail that runs only at generation is a default, not a guardrail.

The tempting version of safety is a checklist you satisfy at the start. You put a strong instruction in front of the model, run a classifier on the output, and call the content safe. Then you build the editor the product actually needs — edit, remove, add, swap an image, regenerate this block — and every one of those buttons is a path around the check you built. Nothing anywhere goes red. This post shows the other version: publishability as a join the compiler cannot execute without, an editor whose every action is a recorded observation, tenancy enforced below the application, and deletion as an orchestrated fan-out with receipts.
01 · Spine02 · Alignment03 · Appropriateness04 · Guardrails & deletion05 · Publishing & fidelity06 · Efficacy

This post deepens the governance layer that wraps every stage — generation, alignment, review and publishing.

Worked exampleThis series uses a K–8 supplemental curriculum publisher as its running example — standards frameworks from three states, generated interactive lessons, the teachers who edit them, and the learning-management courses in two districts the platform publishes into. The pipeline itself is domain-agnostic; the content just happens to belong to third graders.

The gate

Publishability is a join key, not a check you remember to run

Four things have to be true before a block reaches a child: it passed the rubric, a human with the right role approved it, every asset in it has usable rights, and its alignment claim is one the district accepts. Model those as four checks and you have four things to forget. Model them as an inner join and there is no code path that produces a publishable lesson without them.

compile the publishable setapproved at v3 by a reviewer
After approval, the teacher…
This district accepts

↳ Every one of these actions is a feature you shipped on purpose. None of them is malicious.

  • Gate decision, at the current rubric versionthe new block in version 4 has no gate decision yet
  • Review receipt, for this exact versionsigned for version 3, and this is version 4
  • Asset rights, unexpired and usable here14 assets, all licensed for this district, none expired
  • An alignment claim of a class this district acceptsclaim is inferred, and this district accepts inferred
Refused, and nothing checked whether the lesson had been edited. The join to review receipts is keyed by version, and version 4 has no row — which is the single line that makes the editor safe.
sql/publish_compile.sql
SELECT v.lesson_id, v.version, b.block_hash
  FROM lesson_versions       AS v
  JOIN lesson_version_blocks AS vb ON vb.lesson_id = v.lesson_id AND vb.version = v.version
  JOIN blocks                AS b  ON b.block_hash = vb.block_hash
  -- Every join below is a requirement. None of them is a WHERE clause.
  JOIN gate_decisions        AS g  ON g.block_hash = b.block_hash
                                  AND g.rubric_version = $3
                                  AND g.result = 'pass'
  JOIN review_receipts       AS r  ON r.lesson_id = v.lesson_id
                                  AND r.version   = v.version
                                  AND r.role IN ('curriculum_reviewer', 'safety_reviewer')
  JOIN asset_rights          AS ar ON ar.asset_id = ANY (b.asset_ids)
                                  AND ar.usable_in_district = $1
                                  AND ar.expires_at > now()
  JOIN alignment_edges       AS ae ON ae.source_node = 'lesson:' || v.lesson_id
                                  AND ae.edge_class = ANY ($5)   -- district's accepted classes
                                  AND ae.revoked_at IS NULL
 WHERE v.district_id = $1 AND v.env = $2;
Why enforce at publish rather than at generation?
Because generation is one of at least six paths that produce a publishable block, and it is the only one anybody remembers. A teacher can edit a block’s text, delete a block, add one she wrote herself, paste content from a document, swap an image for one she uploaded, or regenerate a single block against a different prompt. None of those is malicious; all of them are features you shipped on purpose. Every one produces content the generation-time classifier never saw. And the mechanism is not “run the classifier again at publish” — it is that publishability is derived from per-block gate decisions, so a block with no decision at the current rubric version simply does not appear in the compiled set. A human-written block is gated exactly like a generated one. There is no privileged origin.

The corollary is an interface requirement rather than a backend one: because approval is keyed to a version, editing an approved lesson must visibly return it to review. A product that lets a teacher edit after approval without changing its state is not saving her a step; it is silently publishing unreviewed content under someone else’s signature.

The editor

Every edit is an observation, and observations are the product

The editor is where the real work happens, and the architectural decision is whether an edit is a mutation or an event. Make it a mutation and you get a simple interface and lose everything. Make it an event and post 3’s entire feedback corpus falls out for free.

src/editor/ops.ts
export type EditOp =
  | { kind: "replace";     position: number; newBlockHash: string }
  | { kind: "delete";      position: number }
  | { kind: "insert";      position: number; newBlockHash: string }
  | { kind: "swap_asset";  position: number; assetId: string; newAssetId: string }
  | { kind: "regenerate";  position: number; instruction: string | null };

export interface EditEvent {
  lessonId: string;          // "lsn_4c19"
  fromVersion: number;       // 3
  toVersion: number;         // 4 — always a new version, never a mutation
  op: EditOp;
  replacedBlockHash: string | null;
  actor: string;             // "usr_2210"
  reason: "too_hard" | "wrong_for_class" | "inaccurate" | "style" | null;   // one click, optional
  at: string;
}

/** Applying an edit is pure: old block list + op -> new block list. The event is the record. */
export function apply(blocks: readonly string[], op: EditOp): string[] {
  switch (op.kind) {
    case "replace":    return blocks.map((b, i) => (i === op.position ? op.newBlockHash : b));
    case "delete":     return blocks.filter((_, i) => i !== op.position);
    case "insert":     return [...blocks.slice(0, op.position), op.newBlockHash, ...blocks.slice(op.position)];
    case "swap_asset":
    case "regenerate": return blocks;   // both mint a new block hash upstream; the list is rewritten there
  }
}
src/editor/assets.ts
/** A swapped-in image is a new asset with its own rights record. No exceptions for uploads. */
export interface AssetRights {
  assetId: string;
  source: "generated" | "licensed" | "uploaded" | "public_domain";
  licenceId: string | null;      // null is only valid for public_domain
  usableInDistricts: string[];   // an explicit list; "all" is not a value
  expiresAt: string | null;
  attributionRequired: boolean;
  reviewedBy: string | null;     // uploads require a human; generated images require a classifier pass
}

The uploaded case is the one teams get wrong. A teacher uploading her own photograph is the most natural action in the editor and the one with the least clear provenance — it may contain a child, it may be someone else’s work, it may be perfectly fine. It cannot be treated as pre-approved because a human chose it; that is precisely the assumption that puts a photograph of a real classroom into a package bound for another district.

Isolation

District and environment are partition keys, and the contractor login is why

Every row and query carries a district and an environment, and storage is namespaced by both. Post 1 defined that type and threaded it through; this post adds the layer underneath it, for when the application is not the thing making the query.

sql/row_policies.sql
ALTER TABLE lesson_versions ENABLE ROW LEVEL SECURITY;

CREATE POLICY district_scope ON lesson_versions
  FOR SELECT
  USING (district_id = current_setting('app.district_id', true)
     AND env         = current_setting('app.env', true));

-- Student interaction data is not merely scoped; contractors cannot see it at all.
ALTER TABLE student_interactions ENABLE ROW LEVEL SECURITY;
CREATE POLICY no_contractor_student_data ON student_interactions
  FOR SELECT
  USING (district_id = current_setting('app.district_id', true)
     AND current_setting('app.role', true) <> 'contractor');
src/security/scope.ts
export function scopeFromClaims(claims: JwtClaims): TenantScope & { role: Role; userId: string } {
  const roles = claims.roles ?? [];
  if (roles.length === 0) throw new ForbiddenError("no roles in token");     // fail closed
  if (!claims.district_id || !claims.env) throw new ForbiddenError("unscoped token");
  return {
    districtId: claims.district_id,
    env: claims.env as "dev" | "prod",
    role: pickHighest(roles),
    userId: claims.sub,
  };
}

An empty role list is a rejection, not a default. The failure you are avoiding is the one where a district’s identity provider stops emitting a claim after a routine upgrade and every token quietly becomes an unscoped one.

Deletion

A deletion that stops at your database is not a deletion

A records request, a parental opt-out, or a district offboarding lands as a tombstone. Deleting the rows is the easy part and the least of it — and one of the six steps below is different in kind from the others.

the fan-out, and its one dead end
Per-lesson publish ledger
Student writing was…

↳ Five of these six steps are ordinary engineering. The sixth is the one you can only get right before the first training run.

Request statuscannot be completed
You can delete the row. You cannot un-train the weights, and you cannot honestly claim otherwise. This is not a compliance edge case — it is a design constraint that had to be settled before the first fine-tune ran, because afterwards there is no remedy.
src/deletion/orchestrator.ts
export async function runDeletion(scope: TenantScope, subject: DeletionSubject): Promise<DsrResult> {
  const receipts: StepReceipt[] = [];
  for (const step of STEPS) {   // ordered: content, telemetry, ledger, LMS, gradebook, corpora
    try {
      receipts.push(await step.run(scope, subject));
    } catch (err) {
      // Do not swallow. The request stays open and visible until every step has a receipt.
      receipts.push({ step: step.id, status: "failed", error: String(err), at: nowIso() });
      await dsrStore.markIncomplete(scope, subject, receipts);
      throw new DeletionIncompleteError(subject.id, step.id);
    }
  }
  await dsrStore.markComplete(scope, subject, receipts);
  return { subject, status: "complete", receipts };
}

When a removal call fails — an expired token, a maintenance window, a course an administrator has locked — you retry with backoff, and until it succeeds the request is not complete. That state has to be visible: an open-request gauge with an age, alerting against your statutory clock, not a swallowed exception in a log nobody reads. A half-finished deletion looks exactly like a finished one from the inside.

The destination you cannot fan out to

If student work ever entered a training corpus or a retained evaluation set, deletion cannot reach it. You can delete the row. You cannot un-train the weights, and you cannot honestly claim otherwise. That is not a compliance edge case; it is a design constraint that has to be settled before the first fine-tune runs, because afterwards there is no remedy. The constraint that makes the rest tractable is boring and absolute: student-authored text never enters a training corpus — not de-identified, not aggregated, not “just for the eval set”. De-identification of free-text writing by children is not a solved problem, and you should not be the team that discovers how unsolved it is. Teacher edits are staff data and are governed rather than forbidden; they are made by adults in a professional capacity, under a contract you can write, which is why post 3’s feedback loop lives there on purpose. And if you must evaluate against real student responses, do it in-session, in memory, against a scoped store with a short retention — and record that you did.

Some destinations also accept a removal and give you no way to verify it landed. You cannot prove content is gone from someone else’s system; you can prove you asked. Record the request payload hash, the response status and the response body as the receipt, and describe it in exactly those terms in your own documentation. Overclaiming here is how a compliance answer becomes a false statement.

Audit

A trail, not a second copy of the content

One append-only line per publish, per gate-decision batch and per deletion. It is deliberately boring.

audit.log
{"ts":"2026-08-25T14:22:07Z","district":"riverbend","env":"prod","actor":"publish-engine",
 "lessonId":"lsn_4c19","version":4,"courseId":"crs_88214","publishId":"pub_7f31",
 "blocksSent":14,"blocksRendered":12,"fidelity":0.857,"rubricVersion":"g3.2026-08",
 "approvedBy":"usr_2210","approvalRole":"curriculum_reviewer",
 "alignmentClasses":["authored","crosswalked"],
 "holdoutSections":2,"experimentId":"exp_fractions_pilot_q1"}
Run it yourself · open-lesson-pipeline @ v4-guardrails
Approve a lesson, edit it after approval, and watch it fall out of the publishable set — then run a deletion and read the receipts.
terminal
git clone https://github.com/xaviramirezcom/open-lesson-pipeline
cd open-lesson-pipeline
git checkout v4-guardrails
pnpm install && uv sync
pnpm lesson up
pnpm lesson seed

pnpm lesson approve lsn_4c19 --version 3 --as curriculum_reviewer
pnpm lesson publish lsn_4c19 --course crs_88214          # 14 blocks published

pnpm lesson edit lsn_4c19 --op replace --position 2 --text "$(cat fixtures/teacher_rewrite.md)"
pnpm lesson publish lsn_4c19 --course crs_88214          # refused: no receipt for version 4

pnpm lesson edit lsn_4c19 --op swap-asset --position 5 --upload fixtures/classroom_photo.jpg
pnpm lesson publish lsn_4c19 --course crs_88214          # refused: asset has no rights record

pnpm lesson dsr --subject student:stu_5512 --district riverbend --print-receipts
The second publish is the whole argument in one refusal: nothing checked whether the lesson had been edited, because the compile-time join to review receipts is keyed by version and version 4 has no row. The third refusal is the same mechanism catching a different mistake — a human-chosen image is still an asset without rights. The deletion run then prints six receipts: content rows, a telemetry mutation id, the ledger rows read, one line per course with the status of the removal, a gradebook reversal, and a final line asserting that no training corpus contains the subject’s text. Kill the mock platform first and the fourth receipt fails, the request is marked incomplete, and the open-request gauge starts ageing against the statutory clock.

Teaching-grade reference implementation, not a production courseware platform. It reproduces the ideas and the queue/warehouse integration shape; bring your own model keys, curriculum data and LMS credentials. LMS adapters run against a local mock by default. MIT-licensed. View the repo →

Explain it back

A district’s safety review passes cleanly. Every generated block was classified, every hold was cleared by a named reviewer, and the audit log is complete. Four months later a parent complains about an image in a fourth-grade lesson — a photograph of identifiable children, which nobody at your company has ever seen. Where did the guardrail leak, and what should have existed from day one?
Reveal a model answer

The image never went through generation, so it never met the classifier. A teacher swapped it in from her own files — almost certainly a photo of her own class, uploaded with entirely good intentions — and the pipeline treated a human choice as its own authorisation. That is the shape of every leak in this post: the guardrail was attached to the generation path rather than to the publish path, and the editor is a supported, encouraged route that bypasses it.

The structural fix is the join. An asset with no rights row is not publishable, full stop, and an uploaded asset requires a human reviewer’s signature before it gets one — which means the upload button’s real cost is a review queue, and if the product cannot afford that queue then it cannot afford the button. Both are honest positions; shipping the button without the queue is not.

Two things should also have existed from day one and probably did not. First, image provenance in the same shape as text provenance: every asset knows whether a model, a licence, or a person put it there, and the publish compiler treats those differently. Second, a per-publish ledger — because the parent’s complaint is about one course and your actual exposure is every course that lesson reached.

The bonus consequence is what makes this urgent rather than merely bad. If that photograph was in a lesson used to build any evaluation or training set, the deletion you are about to run cannot reach it, and no amount of engineering afterwards changes that. The decision that protects you was made — or not made — before the first training run, which is why it belongs in the architecture and not in the policy document.

Next in the series · 05
Publishing & fidelity

Approved content meets an importer you do not control — diffed publishes that do not strand student work, capability descriptors, and the number that says how much of your lesson survived.

Diff, do not republish