Your own activation platform: the spine

End to end

One device, five hops, and the hour that decides them

Before any of the code, watch the thing work — and watch it not work. The trace follows a single device through every stage of the spine. The only control is where the replay stops.

Collection

An event is only trustworthy at the edge you control

Start at the edge, because everything downstream inherits its mistakes. The collector is a first-party server-side endpoint on a subdomain the brand controls — not a third-party pixel on someone else’s domain. It sets the device cookie itself, and because the server sets it, tracking prevention does not cap it at days the way it caps a cookie written from a page script.

• A first-party subdomain, not a pixel. The collector host is a record the brand controls, so the cookie is same-site and the request is not classified as third-party tracking.
• The cookie is HttpOnly. Page scripts cannot read or forge the device id, which means it is something your infrastructure asserts rather than something the page claims.
• The server stamps the time. The client clock is recorded and never trusted — clocks skew by hours across a real device population, and clients lie, both accidentally and on purpose.
• The handler does exactly two things. Validate the envelope, publish it. No lookups, no enrichment, no writes to anything with an index.

import Fastify from "fastify";
import { randomDeviceId, readDeviceCookie } from "./cookie.js";
import { producer } from "../stream/producer.js";
import type { CollectedEvent } from "../contracts/events.js";

const app = Fastify({ logger: true });

app.post("/e", async (req, reply) => {
  const body = req.body as Partial<CollectedEvent>;
  const brandId = String(body.brandId ?? "");
  if (!brandId) return reply.code(400).send({ ok: false });

  // The device id is ours, not the page's: cookie first, mint one otherwise.
  const deviceId = readDeviceCookie(req) ?? randomDeviceId();
  reply.setCookie("_ap_did", deviceId, {
    httpOnly: true, secure: true, sameSite: "lax", path: "/",
    maxAge: 60 * 60 * 24 * 365,
  });

  const event: CollectedEvent = {
    brandId,
    env: (body.env ?? "prod") as "dev" | "prod",
    deviceId,
    sessionId: String(body.sessionId ?? "sess_9c22"),
    name: body.name as CollectedEvent["name"],
    ts: new Date().toISOString(),                        // server-stamped, authoritative
    props: { ...body.props, clientTs: body.ts ?? null }, // recorded, never trusted
    consent: body.consent!,                              // captured at collection time
  };

  await producer.send({
    topic: "events.raw",
    messages: [{ key: `${brandId}:${deviceId}`, value: JSON.stringify(event) }],
  });
  return reply.code(202).send({ ok: true });
});

interface CollectedEvent {
  brandId: string;             // "northwind"
  env: "dev" | "prod";
  deviceId: string;            // "dev_31f0a"
  sessionId: string;           // "sess_9c22"
  name: "page_view" | "identify" | "add_to_cart" | "purchase";
  ts: string;                  // ISO 8601, server-stamped, never client-trusted
  props: Record<string, unknown>;
  consent: ConsentState;       // captured at the moment of collection
}

The pipeline

Five stages, one streaming backbone

Each stage reads a topic and writes a topic. That is the entire architecture, and it is deliberate: any stage can be stopped, rewritten and restarted from its own position without coordinating with the others.

• The resolved topic is compacted on purpose. Keyed by profile, compaction keeps only the latest value per key — so the topic is the current state of the identity graph, not a change feed you have to fold. A new consumer rebuilds full state by reading it from the beginning. No backfill job, no snapshot export, no “please re-run the nightly.”
• Raw is never compacted. The raw event topic is the audit floor. You replay it to rebuild everything else, and you cannot replay history you have compacted away.
• The warehouse is the read side. Events land in a table partitioned by brand and month; traits land in a table that replaces a profile’s row on recompute instead of duplicating it.
• The config store holds definitions, not data. Audience definitions, destination credentials, brand config — small, edited by humans, versioned.

CREATE TABLE events
(
    brand_id   LowCardinality(String),
    env        LowCardinality(String),
    device_id  String,
    session_id String,
    profile_id String DEFAULT '',       -- filled in after resolution
    name       LowCardinality(String),
    ts         DateTime64(3, 'UTC'),
    props      String,                   -- JSON
    consent    String                    -- JSON snapshot at collection time
)
ENGINE = MergeTree
PARTITION BY (brand_id, toYYYYMM(ts))
ORDER BY (brand_id, env, device_id, ts);

What the rest of the series does with this spine, briefly. Post 2 replaces the toy resolver with a real identity graph. Post 3 turns resolved profiles into traits and a propensity score. Post 4 takes the consent record seriously. Post 5 replaces the naive push below with diffing, batching, dead-lettering and match-rate instrumentation. Post 6 asks whether any of it caused revenue that would not have happened anyway. Each owns its stage; none of them re-derive this one.

Isolation

Brand and environment are partition keys, not afterthoughts

Two brands on the same cluster are two different companies, and a test environment must never leak into production. So scope is not a filter someone remembers to apply — it is in the key, the partition, and the namespace.

• The namespace is brand plus environment. Two brands in production do not share a partition, a topic key prefix, or a warehouse part.
• Every stream key is prefixed with the brand. One brand’s traffic spike cannot reorder another’s, and a per-brand replay is a key-range scan rather than a full-topic filter.
• Every table is partitioned by brand. Dropping a brand is dropping partitions. Querying one brand never touches another’s data files.
• Scope is constructed once, at the edge. It travels with the event. Nothing downstream reconstructs it from a header or infers it from a hostname a second time.

export interface BrandScope {
  brandId: string;   // "northwind"
  env: string;       // "prod" | "dev"
}

/** The one place a scope becomes a string. Index prefixes, topic keys, cache keys. */
export function namespace(scope: BrandScope): string {
  return `${scope.brandId}__${scope.env}`;
}

export function eventKey(scope: BrandScope, deviceId: string): string {
  return `${scope.brandId}:${deviceId}`;
}

Partitioning keeps brands apart; it does not keep people safe. Consent as a join key, row policies bound to the executing role, and the external agency login that makes both necessary are post 4’s job.

The sync

The thin end: one audience, one destination

An audience is a document. The compiler turns it into warehouse SQL, the SQL returns hashed keys, and the adapter pushes them at a local mock destination. That is the minimum viable activation, and it is enough to prove the spine end to end.

• Definitions are data, not code. The audience is a document a marketer edits; the compiler is the only thing that turns it into SQL.
• The compiler emits parameters, never interpolated values. An audience definition is user input, and it is treated as such.
• Membership is written back to the stream. Keyed by audience, so post 5’s diffing has something to diff against.
• This push is deliberately naive. Full membership, one batch, no diff, no retries, no match-rate measurement. It is wrong in production and correct as a teaching step.

// A document a marketer edits — data, not code.
export interface AudienceDefinition {
  _id: string;                 // "aud_high_intent_30d"
  brandId: string;             // "northwind"
  env: "dev" | "prod";
  rules: {
    sessions30dAtLeast?: number;   // 3
    ordersLifetimeAtMost?: number; // 0  -> high intent, never purchased
    daysSinceLastOrderAtMost?: number;
  };
  destination: "meta" | "google" | "klaviyo" | "webhook";
  key: "email_sha256" | "phone_sha256";
}

import type { AudienceDefinition } from "./definition.js";

export interface CompiledAudience { sql: string; params: Record<string, unknown>; }

export function compile(def: AudienceDefinition): CompiledAudience {
  const where: string[] = ["t.brand_id = {brandId:String}"];
  const params: Record<string, unknown> = { brandId: def.brandId };

  if (def.rules.sessions30dAtLeast !== undefined) {
    where.push("t.sessions_30d >= {minSessions:UInt32}");
    params.minSessions = def.rules.sessions30dAtLeast;
  }
  if (def.rules.ordersLifetimeAtMost !== undefined) {
    where.push("t.orders_lifetime <= {maxOrders:UInt32}");
    params.maxOrders = def.rules.ordersLifetimeAtMost;
  }

  // Parameters, never interpolated values: an audience definition is user input.
  return {
    sql: `SELECT t.profile_id, i.email_sha256
            FROM profile_traits AS t FINAL
            JOIN identity_keys AS i USING (brand_id, profile_id)
           WHERE ${where.join(" AND ")}
             AND i.email_sha256 != ''`,
    params,
  };
}

Diffing against the last snapshot, batching to a destination’s limits, dead-lettering the rows a success response quietly rejected, and instrumenting match rate all land in post 5. Here, the mock accepts everything and reports nothing — which is exactly the failure mode post 5 exists to fix.

git clone https://github.com/xaviramirezcom/open-activation-platform
cd open-activation-platform
git checkout v1-spine
pnpm install
pnpm activate up            # streaming + warehouse + config store, via docker compose
pnpm activate seed          # synthetic multi-brand traffic, orders and spend

# Replay a synthetic day through the whole spine.
pnpm activate replay --brand northwind --day 2026-08-19

# Follow one device across every hop.
pnpm activate trace dev_31f0a --brand northwind

# Same trace, before the identify event is replayed.
pnpm activate replay --brand northwind --day 2026-08-19 --until 11:00
pnpm activate trace dev_31f0a --brand northwind

Explain it back