Your own lesson pipeline: alignment & the content graph

Worked exampleThis series uses a K–8 supplemental curriculum publisher as its running example — standards frameworks from three states, generated interactive lessons, the teachers who edit them, and the learning-management courses in two districts the platform publishes into. The pipeline itself is domain-agnostic; the content just happens to belong to third graders.

The graph

A unit is a connected component, not a folder

Nodes are typed: a standard, an objective, a lesson, an item, an asset, a district-code alias. Edges are claims — this lesson addresses that objective, this objective decomposes from that standard, this local code means the same as that national one, this objective requires that one first. A unit is whatever set of nodes is reachable through a chosen edge class, and its name is a label for that set rather than a container things live in.

The node types are not interchangeable. A standard is a claim about a child; an objective is a claim about a task; a lesson is an artefact. Collapsing the first two is what makes coverage meaningless, because a standard covered by one objective out of four gets reported as covered.
Edges are observations, not decisions. “The alignment model scored this item at 0.71 against this objective on 14 August under version four” is a fact with a timestamp, and it stays a fact even after you stop trusting that version.
Components are recomputed, never stored as truth. A union-find pass over the claim set gives you units and prerequisite closures in near-linear time, and the answer is a derivation you can reproduce.
Traversal is per edge class. The prerequisite closure of an objective and the coverage set of a standard are different traversals over the same graph. One store, several questions.

the crosswalk collapse

Alias degree cap

An inferred claim may…

↳ Both guards start off, which is what importing a district’s mapping file as-is actually does.

authored — a named specialistcrosswalked — a mapping fileinferred — a similarity score

Components

1

Reported coverage

100%

Uncovered, honestly

none

Refused claims

0

Standards reached, by claim class — never one blended number

authored 3/4crosswalked 4/4inferred 1/4

One component. Every mapping was defensible on its own, and together they made every lesson aligned to every standard. Coverage now reports 100%, which is the most dangerous output in this post — it is the number that gets screenshotted into a district proposal.

And the part that reaches children. A first-grade standard is now in the same component as third-grade objectives, so the prerequisite closure is meaningless and unit sequencing is a coin flip. The coverage number is a business problem. This one shows up as a child meeting the array item before the lesson that makes it legible.

Why store claims and derive units, rather than storing the unit?

Because “these lessons are the third-grade multiplication unit” destroys the evidence. Once you have written that membership you no longer know which claim justified each entry, so you cannot ask whether any particular one was wrong. Store claims with provenance and membership is always a derivation — and anything derived can be re-derived with one claim removed. That is the whole mechanism of a content recall. It is also the only honest answer when a curriculum director asks why an item requiring array reasoning appeared in a unit that has not taught arrays: not “the model put it there”, but “this claim, asserted by version four at confidence 0.71, below this district’s threshold, which we have now revoked along with the four hundred others from that run”.

sql/alignment_edges.sql

-- Append-only. An alignment claim is never updated; it is superseded or revoked.
CREATE TABLE alignment_edges (
  district_id       text        NOT NULL,
  env               text        NOT NULL,
  source_node       text        NOT NULL,       -- "item:itm_9a20"
  target_node       text        NOT NULL,       -- "objective:obj_7d31"
  edge_class        text        NOT NULL,       -- 'authored' | 'crosswalked' | 'inferred'
  confidence        real        NOT NULL,       -- 1.0 for authored
  framework_version text        NOT NULL,       -- a claim is only true of a version
  asserted_by       text        NOT NULL,       -- "usr_2210" or "align-v4"
  source_event_id   text        NOT NULL,       -- provenance, or there is no retraction
  revoked_at        timestamptz,                -- set, never deleted
  ts                timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (district_id, env, source_node, target_node, ts)
) PARTITION BY LIST (district_id);

CREATE INDEX ON alignment_edges (district_id, env, target_node) WHERE revoked_at IS NULL;

Normalisation

Join on a raw standard code and you have invented a second standard

The same standard arrives spelled five different ways — with a framework prefix, without one, with the cluster letter missing, in mixed case, or as a district’s own local code. Nothing downstream will ever tell you they were the same, because none of your joins will error. This is where coverage is lost silently, before a single lesson is generated.

Normalise before you join, both sides, always. Uppercase, strip the framework prefix, collapse separators, and reject rather than guess when the shape is unrecognised.
A missing cluster letter is not a formatting difference. Two codes that look like the same one with a typo genuinely are, in most frameworks — but in some, dropping the letter changes which cluster you are in. So it is a per-framework rule, written down, versioned, defaulting to reject.
District codes are graph nodes, not aliases in a string function. A local code does not normalise to anything. It is a node with an edge to a national code, asserted by a named human — and the difference between those two designs is the whole of the next section.
One normaliser, three call sites. Ingestion, generation and coverage reporting must import the same function. Three implementations means three different frameworks wearing one name.

The failure looks like this: your curriculum ingestion writes the prefixed form, the generator’s objective lookup writes the bare one, and coverage reports zero for a strand you have covered completely. Nobody reports a bug, because nothing errored — someone reports that the third-grade content is missing, and a team spends a sprint regenerating four hundred lessons that already existed.

src/alignment/normalise.ts

export interface CodePolicy {
  frameworkId: string;          // "CCSS-MATH"
  prefixes: string[];           // ["CCSS.MATH.CONTENT.", "CCSS.Math.Content."]
  allowMissingCluster: boolean; // false here; true for frameworks without cluster letters
}

const SHAPE = /^(\d{1,2}|K)\.([A-Z]{2,3})(?:\.([A-Z]))?\.(\d{1,2})([a-z])?$/;

export function normaliseCode(raw: string, policy: CodePolicy): string | null {
  let c = raw.trim().toUpperCase().replace(/[\s_]+/g, "").replace(/-/g, ".");
  for (const p of policy.prefixes) {
    const up = p.toUpperCase();
    if (c.startsWith(up)) { c = c.slice(up.length); break; }
  }
  const m = SHAPE.exec(c);
  if (!m) return null;                                       // reject, never guess
  const [, grade, domain, cluster, num, sub] = m;
  if (!cluster && !policy.allowMissingCluster) return null;   // 3.OA.1 is NOT 3.OA.A.1 here
  return [grade, domain, cluster, num].filter(Boolean).join(".") + (sub ?? "");
}

// normaliseCode("CCSS.MATH.CONTENT.3.OA.A.1", ccss) === "3.OA.A.1"
// normaliseCode(" ccss.math.content.3.oa.a.1 ", ccss) === "3.OA.A.1"
// normaliseCode("3.OA.1", ccss)    === null   // rejected: cluster letter required
// normaliseCode("RBD-M3-04", ccss) === null   // not this framework — it is a crosswalk node

Confidence

Authored, crosswalked and inferred are not the same kind of claim

A curriculum specialist tagging an item is an assertion a qualified human made and will defend. A district mapping is an assertion a different qualified human made about two frameworks. A similarity score above a threshold is an inference you made about them. All three are useful. Storing them in one column is how you lose the ability to ever change your mind.

Authored sits at full confidence. A named human, a timestamp, a framework version. Nothing else earns it.
Inferred is scored and thresholded per district. Below the threshold the edge is not written at all, rather than written-and-ignored. A district that will not accept inferred alignment sets the threshold to one and gets a pipeline that simply produces less, visibly.
Edge class is a separate field from confidence. Marking the layer means it can be revoked, re-scored or disabled per district with one statement, leaving authored history untouched. A confidence band cannot do that, because you cannot tell a human who was unsure from a model that was certain.
Coverage is reported per class or it is not reported. “94% covered” is not a number. “71% authored, 19% crosswalked, 4% inferred” is, and it is the only version a curriculum director can act on.

src/alignment/coverage.ts

/** Coverage is never a single number. A caller that wants one must choose which classes count. */
export interface CoverageBreakdown {
  standardsInScope: number;
  byClass: Record<"authored" | "crosswalked" | "inferred", number>;
  uncovered: string[];        // normalised codes, so the gap list is actionable
}

export const COVERAGE_SQL = `
  SELECT s.code,
         max(CASE WHEN e.edge_class = 'authored'    THEN 1 ELSE 0 END) AS authored,
         max(CASE WHEN e.edge_class = 'crosswalked' THEN 1 ELSE 0 END) AS crosswalked,
         max(CASE WHEN e.edge_class = 'inferred'    THEN 1 ELSE 0 END) AS inferred
    FROM standards AS s
    LEFT JOIN alignment_edges AS e
      ON  e.target_node = 'standard:' || s.code
      AND e.district_id = $1 AND e.env = $2
      AND e.framework_version = s.framework_version   -- a claim is only true of a version
      AND e.revoked_at IS NULL
   WHERE s.framework_id = $3 AND s.grade_band = $4
   GROUP BY s.code`;

Note the join on the framework version. It is one line, and it is the difference between a revision being a migration you run and a revision being a silent, gradual invalidation of every claim you hold. When a district adopts a new revision, every old claim stops matching and coverage drops to near zero overnight — and that is correct, because the claims were about different standards. What you do next is a mapping between versions, authored and reviewed, which is work. The alternative design lets the old claims quietly satisfy the new codes and tells nobody.

The catastrophe

The crosswalk collapse eats a strand and never errors

A district code is mapped to a national standard. Fine. Someone also maps it to the state framework’s equivalent, which is fair. Another district’s importer maps its own code to that as well, and a vendor file maps that to a different standard entirely. Follow the edges and two unrelated standards are in one component. Do it four more times across a strand and the whole domain is a single blob: every lesson aligned to every standard, coverage reporting 100%, prerequisite ordering meaningless, and a unit builder handing a first-grade section an array item.

Alias nodes need a degree cap. Past a small number of distinct framework codes, an alias stops conferring edges entirely — it is a mapping error, not a genuinely universal concept.
Components need a size ceiling per strand. Beyond it the resolver quarantines the component and refuses the merge, rather than writing it and raising a ticket.
The guard refuses; it does not alert. By the time an alert fires, the component has been compiled into a unit and published into forty courses. The wrong children already saw it.
Quarantine is visible, not silent. The refused edge is recorded with its reason so somebody can look, and the pre-guard component keeps serving.

src/alignment/guards.ts

export interface Guards { aliasDegreeCap: number; strandComponentCeiling: number; }

export type Decision =
  | { admit: true }
  | { admit: false; reason: "alias_degree_cap" | "strand_ceiling" | "inferred_bridge" };

export function guard(
  edge: AlignmentEdge, uf: UnionFind, aliasDegree: (n: string) => number, g: Guards,
): Decision {
  for (const n of [edge.sourceNode, edge.targetNode]) {
    if (n.startsWith("crosswalk_alias:") && aliasDegree(n) >= g.aliasDegreeCap)
      return { admit: false, reason: "alias_degree_cap" };   // a mapping error, not a concept
  }
  // An inferred claim may attach a leaf to a component. It may never join two components.
  const bridging = uf.known(edge.sourceNode) && uf.known(edge.targetNode)
    && uf.find(edge.sourceNode) !== uf.find(edge.targetNode);
  if (edge.edgeClass === "inferred" && bridging)
    return { admit: false, reason: "inferred_bridge" };

  const projected = uf.componentSize(edge.sourceNode) + uf.componentSize(edge.targetNode);
  if (bridging && projected > g.strandComponentCeiling)
    return { admit: false, reason: "strand_ceiling" };       // quarantine, do not merge
  return { admit: true };
}

Retraction is the other half. You revoke the offending claim by its source event — set a revoked timestamp, never delete, because the audit trail is the point — recompute the component from the surviving claims, and let it split into however many components the evidence actually supports. Then the part people forget: the old component was published. Every lesson whose alignment changed is live in some set of courses, and a retraction that stops at your database leaves an item sitting in front of children under a claim you no longer make. Which means you must have kept a per-lesson publish ledger to know where it went. That ledger is post 5’s, and this is the first of the two reasons it exists.

Run it yourself · open-lesson-pipeline @ v2-alignment

The seed includes a crosswalk file with the kind of well-meaning over-mapping a real district produces. Resolve twice and watch a strand fall apart into standards.

terminal

git clone https://github.com/xaviramirezcom/open-lesson-pipeline
cd open-lesson-pipeline
git checkout v2-alignment
pnpm install && uv sync
pnpm lesson up
pnpm lesson seed            # two districts, three frameworks, one messy crosswalk

pnpm lesson align --district riverbend --alias-degree-cap off
pnpm lesson graph inspect --node standard:3.OA.A.1     # component size: 47
pnpm lesson coverage --district riverbend --grade 3    # 100% covered

pnpm lesson align --district riverbend --alias-degree-cap on --cap 3
pnpm lesson graph inspect --node standard:3.OA.A.1     # quarantined; 1 standard, 4 objectives
pnpm lesson coverage --district riverbend --grade 3    # 71% authored · 19% crosswalked · 4% inferred

With the cap off, one over-mapped alias welds forty-seven nodes into a single component, and previewing the unit will happily hand you every third-grade operations lesson as one unit whose prerequisite ordering is a coin flip. Coverage reports 100%, which is the most dangerous output in this post — it is the number that gets screenshotted into a district proposal. With the cap on, the alias stops conferring edges past its third distinct code, the component collapses to one standard and its four objectives, the refusals appear in the alignment audit with their reason, and coverage tells you the truth: mostly authored, a fifth crosswalked, and a small inferred tail you can now decide whether to trust.

Teaching-grade reference implementation, not a production courseware platform. It reproduces the ideas and the queue/warehouse integration shape; bring your own model keys, curriculum data and LMS credentials. LMS adapters run against a local mock by default. MIT-licensed. View the repo →

Explain it back

A district’s coverage jumps from 68% to 96% overnight. No lessons were generated, no curriculum was ingested, and no code shipped to the generator. The sales team is delighted and wants to put “96% standards coverage” in a renewal deck. What actually happened, and how would you have caught it before it reached a customer-facing number?

Reveal a model answer

Lesson count did not change; the denominator or the claim set did. The two candidates are a mapping import that added a pile of crosswalked edges — plausible and possibly fine — and an alignment model re-run at a lowered threshold that flooded the graph with inferred edges, which is not. A third possibility is worse and easy to miss: someone ingested a framework revision without a version mapping, so claims from the old version are matching new codes and coverage is counting claims about standards that no longer exist.

Distinguish them in one query, because the class breakdown is stored. If the jump is entirely in inferred, the graph did not learn anything — a threshold moved. You catch that by never reporting a blended coverage number anywhere a human can read it: the API returns a breakdown, the dashboard renders three bars, and there is no code path that produces the single number sales wants, because that number cannot be made honest.

Then enforce it at write time. Alert on the shape of the graph rather than its outputs — the daily distribution of component size per strand, the count of aliases above the degree cap, the ratio of inferred to authored claims per grade — and treat a change there as an incident even when every downstream number looks better. A coverage figure that improves without new content is not good news; it is a claim you did not earn, and it will be in a contract before anyone checks.

The bonus consequence is the one that reaches children. Inferred claims that bridge components also corrupt the prerequisite closure, so unit sequencing silently reorders — and a student meets the array item before the equal-groups lesson that makes it legible. The coverage number is a business problem. The sequencing is the one that shows up as a child deciding she is bad at maths.

Next in the series · 03

Appropriateness as a computed property

Aligned content becomes age-appropriate content — four measurements hiding inside one word, one rubric loaded by two runtimes, and the evaluation that grades its own homework.

Take the word apart →