Diagrams
DataflowFigure
A pipeline: where data comes from, what happens to it, and who ends up with it.
Usage
When to reach for it
Ships from@misoto22/design/diagrams
import { DataflowFigure } from '@misoto22/design/diagrams'Notes
A pipeline: where data comes from, what happens to it, and who ends up with it.
Structurally close to an architecture map and read for a completely different question, which is why it is a separate renderer rather than a preset. An architecture diagram is read for "what talks to what". A data-flow diagram is read for "what is IN this arrow" — and that question is why classification gets its own mono chip under the label rather than being folded into the wording. clickstream / PII touch and clickstream are two different facts, and a governance reviewer is looking for the second one.
Stages are printed as column headings on a rule across the top. They are the axis of the figure: a node's stage is its position along the pipeline, and a reader should be able to answer "how far has this got" by looking up rather than by tracing arrows.
Anatomy
| Element | Description |
|---|---|
| Figure shellrequired | The frame around the picture. What a pipeline hands it is a flow list with the classification folded in — "clickstream — PII touch" — so the fact the diagram was opened for reaches a reader who never sees the chip. |
| Stage headingrequired | One mono caption per stage on a rule across the top. It is the axis: a node’s stage is how far along the pipeline it sits, so "has this been aggregated yet" is answered by looking up rather than by tracing arrows. |
| Node platerequired | One box per node, placed by stage across and row down. The row pitch leaves a clear band between two boxes in the same column, which is where a line and its wording go. |
| Flow | A routed line with a single arrowhead, at its to end. It is the direction claim the whole figure is read for. |
| Classification chip | classification, printed in mono under the wording rather than inside it. "clickstream" and "clickstream / PII touch" are two different facts, and only one of them is what a governance review came for. |
Best practices
Do
- Put the governance fact in classification rather than in the label. It prints as its own chip on the line and is appended to the flow’s summary line, so it survives both a reader who is scanning the picture and one who only has the text.
- Keep stage inside the stages you declared. A node’s x is computed from its own stage index rather than looked up, so a node at stage 5 beside four stages is drawn a full column past the last heading, under no heading at all — development prints DIAGRAM_STAGE_OUT_OF_RANGE, and the summary files it under a band called outside the declared stages rather than beside the nodes the axis does label.
- Write a two-way exchange as two flows. Every line carries a head at its to end only, so one flow between a service and its cache says data moves one way, whatever the label claims about the round trip.
Don’t
- via, channelX and channelY are never honoured here — a data-flow node is always placed by stage and row, so an author’s waypoints refer to a grid this renderer did not build. They are dropped and every line is routed from scratch: the specification still typechecks and the picture is not the one it drew.
- Watch the fan-in. Lines sharing a face are spread evenly across it, and an arrowhead is 11 user units wide against a plate about 55 units tall — so four lines into one face is where the heads meet, and the fifth and sixth arrive as one thick mark. A pipeline at that density wants splitting, not more edges.
Examples
default
Two sources joining at one gate. The mono chip under each label is the classification, printed apart from the wording because a pipeline gets read for what is inside the arrow rather than for what the arrow is called — clickstream and no PII answer different questions, and only one of them is the question a governance reviewer arrived with. The stage headings are the axis, so how far something has got is answered by looking up rather than by tracing.
Event pipeline
4 elements and 3 relationships. What is in the arrow matters more than the arrow.
- Stage: Sources
- Web SDK (frontend) — browser
- Mobile (frontend) — iOS / Android
- Stage: Gate
- Consent gate (security) — policy filter
- Stage: Store
- Warehouse (database) — analytics tables
- Web SDK → Consent gate: clickstream — user events
- Mobile → Consent gate: app events — device events
- Consent gate → Warehouse: accepted — no PII
one source three consumers
A fan-out, and the reason classification is a field rather than a word in the label: one table feeds three consumers and each arrow leaves carrying something different. Three flows share a single face here, so the router spreads their ports across it — stacked on the face midpoint instead, three arrowheads would read as one. The ad partner is the only consumer outside the account, which is why its flow is the one drawn as crossing a boundary.
What leaves the events table
4 elements and 3 relationships. One source, three consumers, three payloads.
- Stage: Source
- Events table (database) — raw
- Stage: Consumers
- BI dashboards (cloud) — internal
- Model training (backend) — nightly
- Ad partner (external) — third party
- Events table → BI dashboards: daily rollup — aggregated
- Events table → Model training: training set — hashed ids
- Events table → Ad partner: conversions — no PII
- Only the ad partner sits outside the account.
- That is the arrow a reviewer reads first.
a stage the data skips
A stage is a claim about how far something has got, and that axis is what makes the bypass legible: the sampled copy leaves the SDK at stage zero and arrives at stage two without passing the redactor, still carrying its PII chip. The same three flows written out as a list of pairs would hide that, because a list has no axis to skip. The emphasised line is what the pipeline is for; the dashed one is a one per cent sample kept so somebody can debug a payload, which is exactly the kind of path an audit finds last.
Clickstream
4 elements and 3 relationships. Three stages, and one copy that skips the middle one.
- Stage: Collect
- Web SDK (frontend) — browser
- Stage: Scrub
- Redactor (security) — drops identifiers
- Stage: Serve
- Lake (database) — partitioned
- Debug bucket (database) — 7-day TTL
- Web SDK → Redactor: events — PII
- Redactor → Lake: hourly — no PII
- Web SDK → Debug bucket: sampled 1% — PII
what is in the arrow
Four stages, because minimisation is a stage: the reviewer's question is whether anything still labelled PII reaches the warehouse, and the answer is a line you follow rather than a paragraph you trust. Every classification is folded into the accessible summary as well as printed on the chip, so a reader who cannot see the picture gets card and email — PII rather than card and email alone. Keep the wording and the classification apart even when they feel like one phrase; the moment they merge, the chip stops being a field anything can be checked against.
Card data, minimised
5 elements and 4 relationships. Read for what crosses each stage, not for what talks to what.
- Stage: Collected
- Checkout form (frontend) — browser
- Support inbox (frontend) — ticket bodies
- Stage: Minimised
- Tokeniser (security) — Vault transit
- Stage: Stored
- Warehouse (database) — order tables
- Stage: Served
- Dashboards (backend) — finance team
- Checkout form → Tokeniser: card and email — PII
- Support inbox → Tokeniser: free text — PII, unstructured
- Tokeniser → Warehouse: order rows — tokenised, no PAN
- Warehouse → Dashboards: daily rollup — aggregated
Accessibility
- A flow’s classification is folded into its summary line, so "clickstream — PII touch" reaches a reader who cannot see the chip.