Your own activation platform: identity resolution

Worked exampleThis series uses a multi-brand customer data & activation platform as its running example — clickstream from direct-to-consumer storefronts, orders, ad spend, and the audiences the platform pushes back out to ad and email destinations. The pipeline itself is domain-agnostic; the data just happens to belong to commerce brands.

The graph

A profile is a connected component, not a row

Nodes are typed keys — a hashed email, a hashed phone number, a device, a login id, an order id. Every event that carries two keys at once produces a link between them. A profile is whatever set of nodes is reachable from any one of them, and the profile id is a name for that set, not a container it lives in.

Nodes are typed keys, not people. A device is a device, not a human; treating it as a person is the mistake that produces every incident later in this post.
Links are observations, not decisions. An identify event saw this device and this hashed address in the same message. That is a fact with a timestamp, and it stays a fact even if you later stop trusting it.
The component is recomputed, never stored as truth. A union-find pass over the link set gives you the answer in near-linear time, and the answer is a derivation you can reproduce.
The customer-facing id is pinned separately. Components merge and grow, and whichever member the solver happens to pick as root changes with insertion order. The public profile id is pinned on first sight so it never flaps under a destination that already knows it.

the merge storm

Device degree cap

Guessed edges may…

↳ Both guards start off, which is what an ordinary upsert-on-email resolver does. Turn them on one at a time.

observed edge — an identify or a purchaseguessed edge — shared network and browser

Profiles

1

Revenue

$1,240

Reported LTV

$1,240

Refused edges

0

One profile. Every kiosk login was a real, observed link, and together they welded four households into a single person carrying four households' order history. Nothing errored — the audience just got strange.

Watch reported LTV rather than the picture. Revenue never moves; only the profile count does. That is the whole mechanism behind “our customers are worth more than we thought” — and behind a bid ceiling raised on nothing.

Why store links and derive components, rather than storing the merge?

Because “these two profiles are now one” destroys the evidence. Once you have written one profile id over another, you no longer know which observation justified it, so you cannot ask whether that one observation was wrong. Store links with provenance and the component is always a derivation — and anything derived can be re-derived with one link removed. That is the whole mechanism of un-merging. It is also the only honest answer when a brand’s legal team asks why two households were treated as one person.

src/identity/union-find.ts

/** In-memory component solver. Production persists this — see below. */
export class UnionFind {
  private parent = new Map<string, string>();
  private rank = new Map<string, number>();

  find(x: string): string {
    const p = this.parent.get(x);
    if (p === undefined) { this.parent.set(x, x); this.rank.set(x, 0); return x; }
    if (p !== x) { const root = this.find(p); this.parent.set(x, root); return root; } // path compression
    return x;
  }

  union(a: string, b: string): boolean {
    const ra = this.find(a), rb = this.find(b);
    if (ra === rb) return false;                       // already the same component
    const [hi, lo] = (this.rank.get(ra)! >= this.rank.get(rb)!) ? [ra, rb] : [rb, ra];
    this.parent.set(lo, hi);
    if (this.rank.get(hi) === this.rank.get(lo)) this.rank.set(hi, this.rank.get(hi)! + 1);
    return true;
  }

  componentSize(x: string): number {
    const root = this.find(x);
    let n = 0;
    for (const k of this.parent.keys()) if (this.find(k) === root) n++;
    return n;
  }
}

The in-memory version is the teaching version. In production the links live in a warehouse table carrying the two nodes, the link class, a confidence, the source event id and a timestamp; a resolver job reads the links touched since its last watermark, expands to the affected components, and writes a component id per node into a table keyed by node. The solver is loaded per affected component, not per brand — you never hold the whole graph in memory, and you never need to.

Normalisation

Hash a raw address and you have invented a second person

Hashing Ana@Northwind.com and ana@northwind.com gives two completely different sixty-four-character strings, and nothing downstream will ever tell you they were the same mailbox. This is where match rate is lost silently, before you have called a single destination.

Lowercase and trim, then hash. Both sides, always, no exceptions for “we control that form”.
Phone numbers get one canonical international format first. A number written with brackets and spaces and the same number written plainly are the same phone and different hashes; destinations expect the canonical form and drop what they cannot parse.
Provider quirks are a policy, not a fix. Some mail providers ignore dots and plus-tags in the local part, so two spellings are one mailbox — at that provider, and only there. Normalising them merges people who really are the same in one place and merges people who are not the same everywhere else. So it is a per-provider rule, written down, versioned, defaulting to off.
One normaliser, three call sites. Collection, resolution and activation must import the same function. Three implementations means three different people wearing one name.

The failure looks like this: checkout posts a shouted address, the newsletter form posts a lowercase one, you get two components, and your match rate is half what the brand’s own list says it should be. Nobody reports a bug, because nothing errored.

src/identity/normalise.ts

import { createHash } from "node:crypto";

export interface EmailPolicy { gmailDots: boolean; plusTags: boolean; }

export function normaliseEmail(raw: string, policy: EmailPolicy): string | null {
  const e = raw.trim().toLowerCase();
  const at = e.lastIndexOf("@");
  if (at < 1 || at === e.length - 1) return null;         // reject, never guess
  let [local, domain] = [e.slice(0, at), e.slice(at + 1)];
  const isGmail = domain === "gmail.com" || domain === "googlemail.com";
  if (policy.plusTags && isGmail) local = local.split("+")[0]!;
  if (policy.gmailDots && isGmail) local = local.replaceAll(".", "");
  if (isGmail) domain = "gmail.com";
  return `${local}@${domain}`;
}

export const sha256 = (v: string): string =>
  createHash("sha256").update(v, "utf8").digest("hex");

// normaliseEmail(" Ana@Northwind.com ", off) === "ana@northwind.com"
// normaliseEmail("a.b+sale@gmail.com", { gmailDots: true, plusTags: true }) === "ab@gmail.com"
// normaliseEmail("a.b@northwind.com", { gmailDots: true, plusTags: true }) === "a.b@northwind.com"
// normaliseEmail("ana@", off) === null

Confidence

Observed links and guessed links are not the same kind of fact

An identify event, or a purchase carrying an address, is an observation the user made themselves. Shared network plus browser fingerprint plus timing is an inference you made about them. Both are useful. Storing them in the same column is how you lose the ability to ever change your mind.

Observed links sit at full confidence. Identify and purchase events only; nothing else earns it.
Guessed links are scored and thresholded. Below the brand’s threshold the link is not written at all — not written-and-ignored.
The class is a separate field, not a confidence band. Marking the whole layer means it can be revoked, re-scored, or disabled per brand with one delete, leaving observed history untouched.
A guessed link may never bridge two components. It can attach a leaf to a family. It cannot marry two families. That rule is one of the two controls on the diagram above.

src/identity/edges.ts

import type { UnionFind } from "./union-find.js";

export type EdgeClass = "deterministic" | "probabilistic";

export interface IdentityEdge {
  brandId: string;
  env: "dev" | "prod";
  nodeA: string;             // "device:dev_31f0a"
  nodeB: string;             // "email_sha256:9f3c..."
  edgeClass: EdgeClass;
  confidence: number;        // 1.0 for identify / purchase
  sourceEventId: string;     // "evt_4a11c9" — provenance, or there is no un-merge
  ts: string;                // ISO 8601, server-stamped
}

/** Probabilistic edges attach leaves; they never join two existing components. */
export function admits(edge: IdentityEdge, uf: UnionFind, known: Set<string>): boolean {
  if (edge.edgeClass === "deterministic") return true;
  const bridging = known.has(edge.nodeA) && known.has(edge.nodeB)
    && uf.find(edge.nodeA) !== uf.find(edge.nodeB);
  return !bridging;
}

The catastrophe

The spreading merge eats a thousand people and never errors

A device in a store is a shared terminal. Eleven customers log in on it over a weekend. Each login is a perfectly good observed link. The device node is now adjacent to eleven identities, the component contains eleven households, and every household inherits the others’ order history. They land in each other’s audiences. Nobody notices until a customer replies to an email about a jacket they never bought.

Device nodes need a degree cap. Past a handful of distinct identities, a device stops conferring links entirely — it is a shared terminal, not a person.
Components need a size ceiling. Beyond it the resolver quarantines the component and refuses the merge, rather than writing it and raising a ticket.
The guard refuses; it does not alert. By the time an alert fires the component has been compiled into an audience and synced to a destination, and the wrong people already saw the ad.
Quarantine is visible, not silent. The refused link is recorded with its reason so someone can look, and the pre-guard component keeps serving.

src/identity/guards.ts

export interface Guards { deviceDegreeCap: number; componentSizeCeiling: number; }

export type Decision =
  | { admit: true }
  | { admit: false; reason: "device_degree_cap" | "component_ceiling" | "probabilistic_bridge" };

export function guard(
  edge: IdentityEdge, uf: UnionFind, deviceDegree: (n: string) => number, g: Guards,
): Decision {
  for (const n of [edge.nodeA, edge.nodeB]) {
    if (n.startsWith("device:") && deviceDegree(n) >= g.deviceDegreeCap)
      return { admit: false, reason: "device_degree_cap" };   // a shared terminal, not a person
  }
  const projected = uf.componentSize(edge.nodeA) + uf.componentSize(edge.nodeB);
  if (uf.find(edge.nodeA) !== uf.find(edge.nodeB) && projected > g.componentSizeCeiling)
    return { admit: false, reason: "component_ceiling" };     // quarantine, do not merge
  return { admit: true };
}

Un-merging is the other half. You delete the offending link by its source event id, recompute the component from the surviving links, and let it split into however many components the evidence actually supports. Each side keeps its pinned id where one existed; new sides get new ones. Then the part people forget: the old component was synced somewhere. Every destination it reached needs an explicit removal, which means you must have kept a per-profile sync ledger to know where it went. That ledger is post 5’s problem, and it is the reason it exists.

Run it yourself · open-activation-platform @ v2-identity

The seed includes a store terminal with dozens of logins on it. Run resolution twice and watch one component fall apart into households.

terminal

git clone https://github.com/xaviramirezcom/open-activation-platform
cd open-activation-platform
git checkout v2-identity
pnpm install
pnpm activate up            # streaming + warehouse + config store, via docker compose
pnpm activate seed          # synthetic multi-brand traffic, orders and spend

pnpm activate resolve --brand northwind --device-degree-cap off
pnpm activate identity inspect --node device:dev_31f0a   # component size: 400

pnpm activate resolve --brand northwind --device-degree-cap on --cap 6
pnpm activate identity inspect --node device:dev_31f0a   # quarantined; back to 1–4 per household

With the cap off, the shared device welds four hundred people into one component — and previewing the high-intent audience will happily hand you all four hundred as a single high-value profile whose lifetime revenue is the sum of a small town. With the cap on, the terminal stops conferring links past its sixth distinct identity, the component collapses into ordinary households of one to four nodes, and the refusals show up as recorded decisions rather than as a support ticket three weeks later.

Teaching-grade reference implementation, not a production customer data platform. It reproduces the ideas and the streaming/warehouse integration shape; bring your own data and destination credentials. Destination adapters run against a local mock by default. MIT-licensed. View the repo →

Explain it back

A brand’s average lifetime value doubles overnight. Revenue is flat, order count is flat, and no code shipped to the pricing or orders pipeline. The marketing team is delighted and wants to raise the target return on ad spend because “our customers are worth more than we thought”. What actually happened, and how would you have caught it before it reached a bid strategy?

Reveal a model answer

Profile count fell; revenue did not. Lifetime value is revenue over profiles, so a merge storm doubles it arithmetically. Almost certainly a shared device — a store terminal, an office network, a returned demo tablet — started conferring observed links across households, and components collapsed into each other. The mechanism is the spreading merge: each individual link was legitimate, and the aggregate was nonsense.

Catch it by instrumenting the shape of the graph, not just its outputs. Alert on the daily distribution of component size — the 99th percentile, the count of components above the ceiling, the maximum device degree — and treat a change there as an incident even when every downstream number looks better. Then enforce it at write time with the degree cap, because an alert is too late once a sync has run.

The bonus consequence is worse than the metric. Merged components inherit each other’s consent state and order history, so someone who never opted in gets marketed to, and a deletion request now has to split a component you already synced to three destinations.

Next in the series · 03

Profiles & propensity: one definition, two paths

Resolved identities become traits and a purchase-propensity score — and point-in-time correctness decides whether that score is real or a very confident restatement of the past.

See the leak →