Versioning: Draft, Release, Promotion

Status: v1.33 — #737: a parameter-set restore tells template_not_found from template_version_not_found by the template entity row, as a pipeline restore does since #733 (§3.5.3). v1.32 — #575: all five wire inventory arms require a RELEASED current pointer (§10.2). v1.31 — #733: a pipeline restore tells template_not_found from template_version_not_found by the template entity row, not a version-1 probe (§3.5.3). v1.30 — #517: a release takes its whole dependency closure, flips consented dependency drafts in one transaction with strict audit, and every pin-affecting write takes the workspace writer guard (§5.3, §3.5, §10.4); a pipeline or parameter-set restore re-judges its outbound pins (§3.5.3). v1.29 — #563: an object whose current pointer names a DRAFT is not promotable and is invisible to a promoter whatever releases it holds (§10.2, §3.4). v1.28 — #459: draft authoring may pin DRAFT or RELEASED children (D58, §3); #462: both pipeline draft purges refuse named parent/dashboard pins before destructive writes. #328: the release flip records the caller output (§5.3). #349: the UI workspace row states the in-page version switch (§7.2); #344: the push closure is read at the pinned versions without the promoter lens (§10.4); #332: the three families audit every lifecycle verb (§7's blanket sentence); #348: §7.2's table disambiguates the REST GET /api/v1/pipelines/{id} from the UI read workspace GET /pipelines/{id}; #10 L1c-c: the count ceiling stated as aggregate; the receive's refused-dashboard rollback witnessed (§10.4); #320: the reverse arrows into other families (§3.5.3); #10: visualizations and dashboards join the lifecycle table (§3.5) Owner: datapipelines.co core Depends on: Pipeline Contract (§13 error catalog, §17 persistence), Templates, Metadata DB (§4.4/§4.5/§4.8/§4.9 — DDL authority), REST API, Pipeline Editor UI Last updated: 2026-10-28

1. Purpose and Scope

This spec defines the version lifecycle for pipelines and templates: how edits become drafts, how a human releases (locks) a draft into an immutable version, and how released versions are later promoted to exactly one higher environment.

It decides:

• The write rule: copy-on-write drafts living in the existing version tables — no separate draft tables.
• The concurrency rule: content-hash preconditions on every mutation — no last-write-wins.
• The release rule: an explicit, UI-only human action. Agents never release.
• The identity rule: version numbers are global identities preserved across environments — imports never renumber.
• The promotion rule: UI-driven, latest-released-only, guarded on both ends.

Everything here applies symmetrically to pipelines (pipeline_versions) and templates (template_versions) unless a section says otherwise.

2. Decision Log

Ratified 2026-08-31, operator + model session:

Ratified 2026-09-08, operator ruling (implemented 099):

Ratified 2026-09-08, operator ruling (implemented 101):

3. Version Lifecycle

Rewritten 2026-09-08 (101, rulings D57–D60). This section replaces the former §3.1–§3.4 in full: the tombstone rule (an executed draft flipping to DISCARDED) and every sentence about deletion are withdrawn. Where prose elsewhere still repeats the old rule, §3.5's table is the ruling.

3.1 Statuses and verbs

Version statuses: DRAFT → RELEASED → DISCARDED.

Five verbs act on versions, and they pair with the statuses exactly:

• Purge is the DRAFT verb (DELETE /pipelines/{id}/versions/{v} — drafts only). The row is hard-deleted together with its executions: development runs of a thing that never shipped are not history. Allocation always reads max(version) + 1, so a purged draft's number is free again exactly when nothing surviving outranks it (its executions went with it, so no dangling reference can exist). Purge is not reversible.
• Discard is the RELEASED verb (POST …/versions/{v}/discard). The row flips to DISCARDED (discarded_at, discarded_by); no dependent may run it, and the row stays.
• Restore is the DISCARDED verb (POST …/versions/{v}/restore): the row returns to RELEASED — its original released_at/released_by return, the discard stamps clear. Before it lands, the pins the version holds are rechecked under the workspace writer guard (§3.5.3, #517): a missing or DISCARDED pin refuses with the family's validation code (400), a present-but-DRAFT one <family>.release.dependency_not_released (409). Discard is reversible; purge is not.
• Release is the DRAFT→RELEASED verb (§5.3, unchanged). Who may: an author or a workspace admin (roles design D8, ratified 2026-09-20: "the author releases, the promoter promotes"). Before R1 (#177) this was the promoter's; the promoter is now an ops role that authors nothing and sees, through the lens (§10.2), only what is promotable (Auth §11A).
• Switch is the pointer verb (§3.4). Who may: an author or a workspace admin — the rollback lever stays with whoever owns the content; the promoter lost it with release.

Who may run the rest: discard, restore and purge are AUTHORING verbs — an author holds all three. That is safe because the system does not allow purging a release: purge is the DRAFT verb, and discard is reversible. Promotion is the promoter's (and the workspace admin's); release is not.

A DRAFT is never discarded and a RELEASED version is never purged: the verbs refuse with pipeline.version.not_released / template.version.not_released and pipeline.version.last_release / template.version.last_release (§13.13/§13.9) rather than guessing intent.

Which posture may run which verb. Discard, restore, purge and the entity purge are authoring writes — a promotion receiver (authoring-enabled=false) refuses them with pipeline.authoring.disabled / template.authoring.disabled, exactly like create, update, release and the draft verbs (§5.5). Two verbs are deliberately NOT authoring: switch (it is the promotion receiver's verb — pointing a receiver at the release it just imported, or back at the one before it, is the human's rollout and rollback lever) and import (unchanged since 039).

3.2 Entity status is derived; names are unique forever

An entity (pipelines / templates row) has no stored lifecycle status. It is derived, never stored:

• ACTIVE while any of its versions is DRAFT or RELEASED;
• DISCARDED when every one of its versions is DISCARDED.

There is no "discard the entity" verb: discarding its last live release IS the entity discard — the derivation lands there on its own. Purging the entity is allowed only when its only version is a DRAFT; the entity row goes with it, in the same transaction, together with the draft's executions. An entity therefore always has ≥ 1 version row unless the entity itself is gone (D57).

Names are unique forever (D59). A discarded pipeline or template keeps its name; restore is refused only while an outbound pin is missing, discarded or not released (§3.5.3, #517) — repair bottom-up, never by editing the released body; no second entity may ever take the name. The V1 is_deleted soft-delete column on both tables is retired (V19): any is_deleted = true row is migrated to "every version DISCARDED" and the column dropped; every reader of is_deleted reads the derived entity status instead. pipeline.validation.pipeline_reference_deleted becomes the derived "the referenced entity is DISCARDED" refusal — it blocks NEW references at save time; existing pinned references keep resolving, exactly as before.

3.3 One draft per entity; number allocation

A partial unique index enforces at most one DRAFT row per pipeline (and per template):

CREATE UNIQUE INDEX uq_pipeline_versions_one_draft
    ON pipeline_versions (pipeline_id) WHERE status = 'DRAFT';

This is what makes the write rule race-safe: two simultaneous first-writers both see "released", both attempt the draft insert — one wins; the loser's insert violates a unique constraint and surfaces as pipeline.version.conflict pointing at the new draft's hash (§4). The loser must re-read and rebase. Which constraint (#276): both writers allocate the same max + 1 below from the same committed rows, so the loser normally collides on the version table's PRIMARY KEY ((pipeline_id, version) / (template_id, version)) before it reaches the one-draft index — Postgres checks unique indexes in creation order, and the key is the older. Every repository that writes a draft (pipelines, templates, parameter sets) maps BOTH to its …version.conflict; mapping the index alone answers the race with a raw driver error (measured for templates by a forced race: TemplateRepositoryIntegrationTest).

Number allocation. A draft pre-allocates max(existing version) + 1 — never current_version + 1, because the pointer can be NULL (D55) and a DISCARDED row keeps its number consumed. The number is stable across all in-place writes to that draft. Numbers held by surviving rows are never reused; a purged draft's number is re-derived from the MAX over what survives (§3.1) — safe precisely because purge deletes the executions with the row.

The draft is allocated the highest number — and an import may land above it. Allocation always reads the MAX (§3.3), so a draft is created above every existing version; the one event that can place a version above a draft afterwards is an IMPORT onto an entity that holds one (§3.5's {D} first-import row lands 1D 2R cur=2): the imported release numbers past the draft, and the pointer names it. The true invariants are the allocation rule and "at most one draft" — the model test asserts both after every event, and NOT a draft-dominance that the table itself overrides.

3.4 current_version is sticky and event-driven (D60)

current_version is the version every pointer-following dependent runs — published endpoints, promotion, schedules (#9 — a schedule's version: "current" follows it, drafts included), dashboards later. It is sticky: it no longer means "the latest RELEASED version" as a derived fact. It moves ONLY on these events, and when it moves it picks the highest-numbered ELIGIBLE live version:

Eligible = RELEASED, plus DRAFT under posture=development (authoring-enabled=true). Under any other posture only RELEASED is eligible — there are no drafts there by construction (§5.5), but the rule is stated, not assumed. A draft's mere EXISTENCE never moves the pointer; a draft is chosen only when the pointer must move and the draft is the highest live version (the discard/purge fallback above). A pointer that NAMES a draft has one promotion consequence: while it stays there the object is not promotable and is invisible to a promoter whatever releases it holds (§10.2, #563).

Dependents read the pointer; authors read "last" (D56, unchanged). Manual execute with no version is NOT a pointer read: it runs the working version — the draft when one exists, else the latest release — resolved in one place (PipelineService.workingVersion). The published surface serves the pointer when it names a RELEASED version, which under hardened postures is the only thing it can name.

NULL pointer. A published endpoint answers endpoint.pipeline_not_released at serve time — the endpoint row is not deleted, and an entity whose status is DISCARDED answers the same refusal rather than a 404; a schedule (#9) records a not_started run (reason pointer_null) and blocks until a person unblocks it (Scheduler §5); promotion has nothing to push.

A draft pointer (D63, 2026-09-09). Under the development posture the pointer may name a DRAFT (the D60 fallback, or a manual switch), and a published endpoint serves it — the endpoint has to be testable before the release, so in development the API can be pointed at any live version. Outside development no draft exists, so a release is served by construction; the serve path checks the same eligibility rule the pointer uses (PipelineVersionStatus.eligibleForPointer), never a status of its own.

3.5 The lifecycle table

The tree. Pipelines and templates are two folder trees — joined by #194 lane D's third, parameter sets (the record's §8.1: the templates' table shape, the pipelines' hash and index semantics, addressed by id). Every lifecycle row below reads with parameter.* codes for the set family; a set releases only when every pinned template version is RELEASED (or the 142 cascade with consent), and release re-runs the record's §4 source validation against the pins as they are now. #10 adds the fourth and fifth families — visualizations and dashboards (Dashboards §3, V42: the parameter-set shape twice) — and every row reads with visualization.* / dashboard.* codes for them. A visualization releases only with at least one test case, its transform pin RELEASED (or the 142 cascade with consent) and the D56 evidence gate passing (the tests lane's; until it is installed every visualization release is refused); a DRAFT visualization version pinned by a LIVE dashboard version is never purged, nor a pinned release discarded (visualization.version.pinned — graph rule 1). A dashboard releases only when its set is RELEASED, its sources RELEASED and read-only, and every pinned visualization RELEASED — or released with it in one transaction under release_pinned_visualizations (D61). Every leaf has an ordered version list. Dependencies are edges: exact-version pins (a pipeline node's template pin, a PIPELINE node's child pin) and pointer edges (endpoint→current, schedule→current). Three graph rules cover every refusal:

1. A version with an inbound exact pin from a LIVE (non-discarded) version cannot be discarded or purged — the refusal names the pinning entities. Codes: pipeline.version.pinned for child pipelines and, since #320, for the dashboards whose sources pin the version (details.referencing_dashboards, beside pinned_by); for templates the existing template.in_use IS the in-use refusal — no template.version.pinned is added — and it names pipelines (pinned_by), parameter sets (referencing_parameter_sets) and, since #320, visualizations whose transform.template pins the version (referencing_visualizations); parameter.in_use (#320) for a parameter set a dashboard pins; visualization.version.pinned for a visualization a dashboard pins. Every arrow, verb and details key is §3.5.3.
2. A pointer edge follows the pointer and refuses on NULL (§3.4).
3. An entity can be purged only when it has no inbound edges of any kind and its only version is a DRAFT. For pipelines both draft purge entry points and the explicit entity purge ask the existing entity guard: live parent pipeline versions pinning the exact draft, plus every stored dashboard version of a dashboard that still has a live version, including DISCARDED versions, pinning any version of that pipeline. There is no historical parent-pipeline scan. A never-released pipeline cannot be published, but since #459 draft parents and dashboard sources may pin it. On the template side this is the template.in_use check. A draft purge that takes the entity with it (the draft is the only version) is an entity purge and asks that entity question; a draft purge that keeps other versions asks rule 1's live exact-pin question of the draft's own version. Both pipeline draft purges refuse pipeline.version.pinned with pinned_by and referencing_dashboards before deleting executions or other rows (#462).

Composition (D58, #459). A draft pipeline may pin a live DRAFT or RELEASED child version at save; a DISCARDED or missing child is refused. Release and import require every pinned child RELEASED (pipeline.validation.pipeline_reference_not_released), so released composition still references only reviewed content. Template pins keep their rule: a DRAFT pipeline may pin a DRAFT or RELEASED template version while iterating, and release requires every pinned template version RELEASED (§5.3). Pinning a DISCARDED template version is refused at save (pipeline.validation.template_version_not_found) — which is what makes "no live version pins a DISCARDED version" an invariant rather than a hope.

The sole version (D57). Discarding the only RELEASED version is legal — the entity becomes DISCARDED and the pointer goes NULL; restore brings it back. The refusal pipeline.version.last_release / template.version.last_release belongs to the PURGE path only: "discard is per version, the entity stays; restore or release something first."

Purging a draft pipeline offers its exclusive draft templates. The purge call accepts include_exclusive_draft_templates: true: the service computes the set of DRAFT template versions the draft body pins that no OTHER stored pipeline version pins — DRAFT, RELEASED or DISCARDED alike (R12, 2026-09-13: a discarded version can be restored and must keep what it runs; a purged version has no row and cannot pin) — and either purges them with the pipeline (true) or merely returns them in the response (false, the default) — either way the caller sees the set, so the UI (102's entity-purge dialog, §4.3d) offers the cleanup as a checkbox and the default is the same offer the response carries. Since #320 "no OTHER pinner" spans every aggregate that pins templates: a DRAFT-only template that a parameter set or a visualization also pins (any version, any status) is never offered and never purged — the pipeline purge proceeds, and the response's kept_draft_templates names each template it kept and who else pins it (referencing_parameter_sets, referencing_visualizations, the keys template.in_use uses). The offered purge re-checks inside the purge's transaction and refuses template.in_use — rolling the whole purge back — if such a pin appeared since.

3.5.1 Notation and blanket rules

Rows below use: 1R 2X 3D cur=1 — version number with status letter (R released, X discarded, D draft), cur the pointer (∅ = NULL). The Shape column groups rows into the ten families {D} {R} {R,D} {R,R} {R,R,D} {R,X} {X,D} {X,X} {X,X,D} {R,X,D}; within a family the order of statuses matters for pointer outcomes, so every row states its full Before map. Codes shown are the pipeline spelling; the template twins substitute template.version.last_release / not_released / not_discarded / not_eligible and template.in_use (for pinned).

• Posture blanket rule. Discard, restore, purge and purge(entity) are authoring writes: under hardened posture every such row — whatever its shape — is refused pipeline.authoring.disabled / template.authoring.disabled before any other check. The rows below give the development outcome for those verbs, and carry a posture cell only where a non-authoring verb (switch, import) differs by posture: dev = development; hard = hardened; both = identical under either. Draft-holding shapes cannot arise under hardened (§5.5); their rows state the rule.
• Missing versions. An event naming a version the entity does not hold is 404 pipeline.execution.not_found (template.not_found for templates) — uniformly, not per-row.
• The pointer-followers. After every allowed row, cur names a live eligible version or is NULL; dependents behave per §3.4. This is the invariant the model test asserts after every event — the table never restates it per row.

3.5.2 The table

VersioningSpecDriftTest parses this table (the ScopeMatrixSpecDriftTest shape), so the model test in §13 runs the DOC's rows, not a copy. Editing a row is editing the rule; the parser fails the build on a row it cannot read.

Release across entities, and the twins (#517). The table models ONE entity, so a release that flips dependencies appears only as the release(releasePinnedTemplates) row — the legacy alias, its template twin releasing with it. A release plan (§5.3: tree_hash, include_dependencies) is the same transition applied to every selected DRAFT in the closure, dependency-first, in one transaction: each selected node takes its own release row's outcome, and any refused row refuses the whole release. The template twin's release row runs in one metadata transaction under the workspace writer guard since #517 (it was a bare statement before). Every other event in the table — and both import rows, first and subsequent — is a write that changes what a release resolves, so it too runs under the writer guard; a guard wait past lock-wait-seconds refuses the row with pipeline.release.busy / template.release.busy, nothing written. The table's outcomes are otherwise unchanged.

3.5.3 The reverse arrows into other families (#320)

A visualization and a dashboard pin other artifacts by name and exact version (the design record §4.2: "deletion and purge guards keep referenced releases"). The scans live over visualization_versions and dashboard_versions (ArtifactDependents, workspace-scoped, one bound JSONB probe per arrow); the owning modules reach them through a port application implements — no module's SQL names another module's tables.

Endpoints and schedules pin nothing (pointer edges, rule 2): no arm exists for them.

Which question each verb asks. A discard and a draft purge that keeps other versions ask the LIVE exact-pin question (rule 1: DRAFT or RELEASED dependent versions pinning exactly this version). An entity purge, and a draft purge that takes the entity, ask the ANY-version question (rule 3, R12: every stored dependent version — DISCARDED included — of a dependent that still has a live version, because a restore would resurrect the pin). The two questions share one statement shape and differ only in their scope predicate.

Pipeline draft pins are reachable (#459, #462). Draft parents and draft dashboards may save pins to a DRAFT pipeline. A sole-draft purge therefore asks the entity guard before any deletion; a nonsole purge asks live exact pins. Dashboard history can still contain a draft pin beside a repointed live version, so the sole-draft dashboard scan includes that stored history. Parent pipeline pins remain live/exact in both modes; this does not add a historical parent scan. A RELEASED dependent still requires RELEASED dependencies (or a consented cascade at release), and releases never return to DRAFT or become purgeable. Release/import strictness is unchanged. The template and parameter-set entity guards retain their any-version scans; the visualization entity guard counts LIVE dashboard versions only.

Restore re-judges dependencies (D7). A DISCARDED version protects nothing, so the version it pins can be discarded while it is discarded. dashboards' and visualizations' restore therefore re-runs the dependency rules against today's state and refuses dashboard.validation.dependency_not_found / visualization.validation.transform_binding_invalid (reason: template_not_found or template_version_not_found), naming the dead pin — only the dependency failures; a rule that tightened since is not a dangling pin.

Pipelines and parameter sets do the same since #517 (release-dependencies record §12.1, ruled 2026-10-25), under the workspace writer guard and from metadata alone (no render, no check gate, no selector dry run, no customer connection). A missing or DISCARDED pin refuses with the family's validation code (400): a template pin template_not_found / template_version_not_found (reason: discarded when the version is DISCARDED; for a pipeline's template pin and a parameter set's selector pin alike, the template ENTITY row of the workspace tells a missing template from a missing version — a template can lose version 1 to a preserved-number import or a draft purge, so no version number stands in for the entity, #733, #737), a pipeline's child pin pipeline_not_found / pipeline_version_not_found / pipeline_reference_not_released (status: DISCARDED). A present-but-DRAFT pin refuses <family>.release.dependency_not_released (409, the import-landing precedent, reason: not_released in the shared blockers shape); no save rule lets a once-released body pin a draft, so that branch is defensive. The dashboard and visualization restores have no such branch: they refuse the missing pin only and admit a present-but-DRAFT one. The 400 answers first. Repair is bottom-up: restore the dependency, then the dependent. A refusal writes nothing — status, pointer and audit stand.

The lens. templates_used_by and the template screen narrow the visualization arm like the others: a hidden visualization is never named, and a narrowing view reports RELEASED rows only. The refusals need no lens: every verb above is held by author, workspace admin and super admin (and their MCP keys) — a promoter holds none of them (auth §7.6), so no lensed caller can reach a refusal. templates_purge_draft still narrows its echo (a name never, a count never — only pins_hidden: true).

The residual window (the C32 shape). The dependents are read by the service before the write, not inside the statement: a dashboard RELEASE that races a pipeline DISCARD can leave a released dashboard pinning a discarded release. The runtime then refuses to serve it (dashboard.runtime.dependency_missing), and a re-run of validate names the dead pin. The pipeline draft purge preflight also retains a concurrent pin/write window: a draft reference saved after the scan can race the purge. This service check is not a concurrency-proof referential constraint; if such a race removes the entity, later name reuse can still re-resolve a dangling draft pin.

3.6 The one write rule (copy-on-write)

When we write, we make sure it is not a released version. If it is released, we first create a draft out of the released version, then write. Otherwise we write — no problem.

Mechanically, PUT /pipelines/{id} resolves the current state of the pipeline and takes exactly one of two branches (§5.1/§5.2). There is no third branch: a PUT never appends a released version, and a PUT never touches a RELEASED or DISCARDED row.

Creation is draft-first too, without exception (D55). There is nothing to copy from, so there is no copy-on-write step — but POST /pipelines, pipelines_create, POST /templates and templates_create all land version 1 as DRAFT, and the index row's current_version stays NULL until a human releases it. DRAFT → RELEASED is a human step; D4 holds with no exception, on creation as on every subsequent change.

The pipeline is executable the moment it is created all the same: drafts have been executable since 039 (§8), and execute with no version runs the working version (§7.2). That is what retired the previous rule. Until 2026-09-08 this section said creation "lands version 1 directly as RELEASED, so an MCP-authored pipeline is executable the moment it is created" — an assumption written on 2026-08-31, never ratified, whose only justification stopped being true when drafts became executable. Its cost was real: an agent that created demo/top_carrier_by_borough followed it to the letter and produced a RELEASED pipeline no human had looked at.

The two create paths that are not authoring keep landing RELEASED, and that is the whole exception: a promotion import (§9.2) and the seeders that ride the same import services (ExampleContentSeeder, LakeBootstrapSeeder). The content they write was reviewed and released where it came from, and the system actor is not an agent asking for a review. The distinction is a required argument on both repositories' create (CreateLifecycle), so a new create path has to say which of the two it is and a missed one is a compile error.

3.7 What rides the draft vs. the release

name, display_name and description exist in two places, and that is deliberate — they are not two copies of one fact:

• pipeline_versions.body_json is the artifact. The contract requires those fields (§3.1) and environment-portability requires the body to stand alone (§2 principle 1): an export bundle that did not describe its own pipeline could not be imported.
• The pipelines row is an INDEX over the artifact. UNIQUE (name) needs a real column; the list screens' search filters PipelineRecord.name/displayName/description directly; every join reads the row without parsing JSONB.

Ratified 2026-08-31 — the row indexes the CURRENT RELEASED body. Metadata edits ride the release: a draft carries its metadata in its own version row, and the pipelines row keeps the released values until lock. This is not a preference between two workable options. Populating the row from a draft would make it an index over a mixture — some rows describing released content, some describing unreleased edits, with nothing in the schema marking which — and every consumer of the row (search, list, name lookup) would silently inherit that ambiguity.

The workspace header shows a draft's pending-release affordance when the caller's view admits one (the Release / Purge-draft verbs are the lifecycle dialogs', role-gated); the read page's version chip names the version it RESOLVED — draft or release — and the list shows the released name until lock.

What surface the write arrived on (102, owner ruling 2026-09-09). Every version row on both tables carries created_via / updated_via — 'session' | 'api_key' | 'mcp', stamped at the ENTRY POINT (the REST controllers map the principal's auth method; the MCP tools pass mcp; import and seed paths keep the 'session' default) — while created_by / updated_by keep naming the PERSON on every path, key principals included (a key's writes are its owner's): no key id is ever stored on a row. A release stamps nothing new — D4 already says a release is a human in a session, which is asserted, not recorded.

Draft-write-time uniqueness check (ratified, and the reason this is safe). Because UNIQUE (name) lives on the row and the row is not updated until release, a draft rename to a taken name would otherwise fail only at Release — after the work is done. So a draft write that changes name validates it immediately:

SELECT 1 FROM pipelines
 WHERE name = :newName AND workspace_id = :workspaceId AND id <> :thisPipelineId;

A hit is pipeline.validation.duplicate_name (409 — the existing code, no catalogue addition) at draft-write time. Uniqueness is checked AGAINST the index without being enforced FROM it, so the early failure costs nothing structurally. The release-time constraint remains as the backstop: it is the authority, this is the courtesy.

4. Content Hash (body_hash)

4.1 Definition

body_hash = SHA-256 (hex) of the canonical body JSON, stored on every version row at write time. Canonicalization is frozen — the hash is only as good as the canonical form — and v1.3 pins it mechanically:

• Pipelines: the canonical form is the database's JSONB text projection of the body — encode(sha256(convert_to(body_json::text, 'UTF8')), 'hex'), computed BY THE DATABASE in V6's backfill and in every repository write, one expression everywhere. The serializer's output remains the write FORMAT, but a JSONB column does not preserve the writer's key order, so hashing the serializer string in the application while the backfill hashed the stored projection would give two hashes for one body — exactly the "every pre-existing row fails its first precondition check" failure §11's backfill option exists to avoid. Database computation is also what makes §11's own "pgcrypto digest" backfill option sound.
• Templates: the canonical form is the version-owned field object {engine, dialect, is_library, imports, body} projected through jsonb_build_object(…) (which normalizes key order deterministically) and hashed with the same expression. display_name/description are NOT in it — see §6's asymmetry note.

The stored hash doubles as:

1. The precondition token for every mutation (§4.2).
2. Cross-server content identity for promotion delta detection (§11.2) — same content hash on two servers ⇒ same body, regardless of environment.
3. A cheap stored-integrity check (recompute vs. stored, e.g. in audits).

4.2 The precondition protocol

Every mutation — draft create, draft write, release, discard — carries the hash of the version the caller based its change on (an If-Match-style request header or an explicit expected_hash field; the REST contract fixes the exact spelling at implementation). The server evaluates the precondition inside the mutating statement's WHERE clause:

• Draft write / release / discard: expected hash must equal the DRAFT row's stored body_hash.
• Draft create (first write after release): expected hash must equal the RELEASED row's stored body_hash.

Zero rows affected ⇒ the base is stale ⇒ 409 pipeline.version.conflict (or template.version.conflict) with the current state in details:

{
  "code": "pipeline.version.conflict",
  "message": "Pipeline was modified by someone else after you loaded it.",
  "details": {
    "current_body_hash": "…",
    "current_status": "DRAFT",
    "updated_by": "user-id",
    "updated_at": "2026-08-31T14:03:11Z"
  }
}

The client's recovery path is explicit: reload, diff against its own edit, re-apply. The server never merges and never overwrites. This protects every pairing — two engineers, engineer + MCP agent, two tabs.

4.3 What the hash does NOT do

It is not a diff format, not a merge token, and not a security signature (it is not keyed). Its only job is equality: is the thing I am about to modify the thing I looked at?

5. Write Paths

All four paths stay single-statement data-modifying CTEs — the repository house pattern (PipelineRepository KDoc) — so each remains atomic without an enclosing transaction. Sketches; the implementation owns final SQL.

5.1 Draft create (copy-on-write, first write after release)

WITH guard AS (                       -- precondition: caller's base is the released body
    SELECT 1 FROM pipeline_versions
     WHERE pipeline_id = :id AND version = :currentVersion
       AND status = 'RELEASED' AND body_hash = :expectedHash
), draft AS (
    INSERT INTO pipeline_versions
        (pipeline_id, version, body_json, body_hash, status, created_by, updated_by, updated_at)
    SELECT pipeline_id, version + 1, CAST(:bodyJson AS jsonb), :bodyHash, 'DRAFT', :actor, :actor, NOW()
      FROM pipeline_versions
     WHERE pipeline_id = :id AND version = :currentVersion AND status = 'RELEASED'
       AND <the incoming body's hash expression> <> v.body_hash   -- the no-op guard (below)
       AND NOT EXISTS (SELECT 1 FROM pipeline_versions
                       WHERE pipeline_id = :id AND status = 'DRAFT')
    RETURNING pipeline_id, version
)
SELECT * FROM draft, guard            -- guard empty ⇒ 0 rows ⇒ 409

The no-op guard (v1.5): a PUT whose body is identical to the released one must not create a draft, burn a version number, or light the pending-release badge for a change nobody made. The predicate compares the incoming body's hash to the released row's stored hash, in the same statement, by the same canonical-hash expression the INSERT would store — never in the application, where a second implementation of the canonical form is exactly the defect 035 found live. The statement's no-op arm returns the current RELEASED detail in that case, so the response for a no-op shows the current state — status: RELEASED, no draft — rather than making the caller infer it from an absence. It is not a 4xx: the write was well-formed and the outcome is "already in that state". Consequences the implementation pins:

• Both the draft arm and the no-op arm join the guard, so a stale precondition still yields zero rows ⇒ 409 — D3's no-last-write-wins outranks tidiness.
• The no-op arm requires that no draft exists: identical content written while a draft is open is a stale base (409 carrying the draft's state), never a "no draft" answer that would contradict the working state.
• A draft edited back to match its released parent is LEFT ALONE (written in place, never auto-discarded): silently deleting a draft row, its version number and its updated_by history because someone reverted would be surprising, and discard stays explicit.
• Templates mirror all of this (§6), with the documented asymmetry that display_name/description are not part of the hashed artifact — a content-identical save that changes only index metadata still moves the metadata, without opening a draft.

Why this matters beyond tidiness: a draft exists if and only if the content genuinely differs. Draft-existence is therefore a truthful signal — which is what §7's authoring reads rely on and what two people editing one pipeline need.

5.2 Draft write (in-place)

UPDATE pipeline_versions
   SET body_json = CAST(:bodyJson AS jsonb), body_hash = :bodyHash,
       updated_by = :actor, updated_at = NOW()
 WHERE pipeline_id = :id AND status = 'DRAFT' AND body_hash = :expectedHash
RETURNING version                     -- 0 rows ⇒ 409 (stale base or no draft)

A PUT arriving while no draft exists takes §5.1; a PUT arriving while a draft exists takes §5.2. The branch is decided by the draft's existence, and both branches carry the same precondition semantics.

5.3 Release (lock)

One statement, three effects — validate happens in the service layer immediately before:

WITH locked AS (
    UPDATE pipeline_versions
       SET status = 'RELEASED', released_at = NOW(), released_by = :actor,
           caller_output_json = (              -- #328: the D1 record, in the SAME statement
               SELECT e.result_schema_json
                 FROM pipeline_executions e
                WHERE e.pipeline_id = :id
                  AND e.pipeline_version = pipeline_versions.version
                  AND e.status = 'SUCCESS' AND e.parent_execution_id IS NULL
                  AND e.result_schema_json IS NOT NULL
                  AND e.started_at > COALESCE(pipeline_versions.updated_at,
                                              pipeline_versions.created_at)
                ORDER BY e.started_at DESC LIMIT 1)
     WHERE pipeline_id = :id AND status = 'DRAFT' AND body_hash = :expectedHash
    RETURNING version
), bumped AS (
    UPDATE pipelines
       SET current_version = (SELECT version FROM locked),
           name = :name, display_name = :displayName, description = :description,
           updated_at = NOW()
     WHERE id = :id
    RETURNING current_version
)
SELECT * FROM locked, bumped          -- 0 rows ⇒ 409

(v1.3 dropped the sketch's updated_at = NOW() from the flip: §11's column note makes updated_at draft-write metadata — a release or discard does not restamp it, so a released row keeps the timestamp of its last draft write. The two sections now agree.)

(#328, v1.26: the flip also records caller_output_json — the D1 read, sketched above. The ordering predicate is exactly the draft-write clock §11 pins: updated_at moves on every draft write and never at release, so "started after the last draft write" reads precisely "ran THIS body". A never-run draft releases with a NULL record; the response and audit row say caller_output = not_observed and the release is never refused for a missing run. The record's shape, bounds and the reader's precedence are pipeline-contract §3.3.1.)

Preconditions, evaluated server-side before the statement runs:

1. The expected hash matches the draft (§4.2) — you release what you tested.

2. Every dependency in the draft's release closure is RELEASED, or flips in the same release — dependencies lock first (#517, release-dependencies record §1, §4). The closure is every template version the body pins and every child pipeline version a PIPELINE node pins, transitively, enumerated within the configured bounds (configuration §3.35: depth, versions, edges, wire bytes — over a bound refuses pipeline.release.dependency_limit, a cycle pipeline.release.dependency_cycle). A dependency that is not RELEASED and not flipped by this request refuses pipeline.release.dependency_not_released in the shared blockers shape ({root, blockers:[{node_id, family, name, version, status, reason, paths}], blockers_truncated}, REST §5.10). (Pinning a DRAFT dependency from a DRAFT pipeline is legal while iterating — §6 — and only becomes an error at release time.) A DISCARDED or MISSING pin is never releasable; a hidden pin answers exactly as an absent one.

   What the request consents to flip. Explicit consent, never implied; the default is the refusal above.

   • No consent — only the pipeline flips.
   • The legacy alias (142, owner 2026-09-15: "a headache-free promotion") — releasePinnedTemplates = true, ?release_pinned_templates=true over REST, the checked-by-default "Also release these N draft templates" group in the release dialog. It selects exactly the root's DIRECT DRAFT template pins, each released at its PINNED version and nothing newer; a deeper DRAFT is a blocker. Its success response and audit keys are release 142's.
   • A release plan (#517) — tree_hash (the preview digest of the closure the caller reviewed), include_dependencies (the DRAFT nodes it consents to flip; every reachable DRAFT must be selected) and optionally check_overrides (a node's own release-check override reason). The digest is compared before any validation or check work, and again inside the transaction under the writer guard; any drift between preview and submit — a body changed in place without a version change included — refuses pipeline.release.plan_conflict. The snapshot includes the resolved entity UUID as well as the body hash; purging and recreating an identical name/version cannot reuse consent. Native preparation reads the body and its database hash together; validation, checks, released entity metadata and the exact-version CAS use that same body. The caller must hold each flipped family's release permission.
   • The arming rule — the alias together with include_dependencies is pipeline.release.plan_invalid (two authorities); any plan field without tree_hash is 428 pipeline.release.plan_required. The record's 428 for a BARE alias (§17 row 6) is armed by one constant, LegacyReleaseAlias.LEGACY_ALIAS_WITHOUT_PLAN_ADMITTED, which stays true until the release dialog sends tree_hash (#522): until then a bare alias is the legacy cascade above, unchanged.

   One transaction, strictly audited. Every release — consent or none — runs in ONE metadata transaction: the workspace writer guard first (below), then the dependencies flip dependency-first, each at the hash the closure read, the pipeline last; one *.version.released audit row per actual flip is written in that same transaction, and an audit store that refuses the row refuses the release. A refusal anywhere — a dependency's own preconditions, a stale hash on the flip, a concurrent write, the audit row — rolls the whole transaction back, so no dependency is ever left released with the pipeline still a draft. Each cascaded row keeps the direct verb's shape (a template's cascade_from_pipeline_id / cascade_from_version) plus cascade_root; every row of the release shares release_id and carries scope: "version"; the pipeline's row keeps templates_released: [{template_id, version}] (an empty list when nothing cascaded) and gains dependencies_released. Each dependency flips by compare-and-set at the exact version and hash the closure read — the pin names the exact version, so there is no second hash handshake per dependency. The release check gate (precondition 4) runs before the transaction, for each flipped node against its draft. Promotion receive is unaffected: promoted content arrives RELEASED (§10), so no consent is exposed there. The same role holds both verbs (auth §7.6, "Release a version"), so consent grants no new capability; the MCP surface is untouched — no tool releases anything.

   The workspace writer guard (#517, record §4.3). A release must judge a closure no concurrent write can change under it. Every metadata write that changes what a release resolves — a pipeline, template, parameter-set, visualization or dashboard save, discard, restore, purge, manual switch or import, and a visualization test session's open and submit — runs in a metadata transaction that first locks the workspace's own workspaces row (SET LOCAL lock_timeout from the server-validated lock-wait-seconds, then SELECT … FOR NO KEY UPDATE). Writers in one workspace serialise; other workspaces never wait. A wait past the bound answers the writer's family <family>.release.busy and nothing is written.

3. Full pipeline-contract §12 validation re-runs on the draft body. Release is the final save-time gate; nothing is released that the validator would refuse.

4. Every release check the body carries passes (140). When the draft body declares checks[], the server runs them NOW — a fresh run, via = release, one pipeline_check_runs row per check — BEFORE the flip, and outside the metadata transaction (the runs open customer-datasource connections). Any fail or error verdict refuses the release with 409 pipeline.check.failed, whose details.checks lists every failing check with its expected and observed values and the run message — UNLESS the request carries override_checks_reason, a non-blank string of at least 10 characters, in which case the release proceeds and the pipeline.version.released audit event gains checks_overridden: [ids] and override_reason.

Checks are opt-in: a version with no checks[] releases exactly as before. The agent never supplies an observed value — it writes the query and the expectation; only the server's own run produces observed, so the gate cannot be talked past.

5.4 Purge the draft (was "discard", 101)

The draft verb is purge (§3.1): the row is hard-deleted together with its executions — development runs of a thing that never shipped are not history, and the executions FK must never force a tombstone. The service deletes the execution rows in the same transaction (the constraint stays NO ACTION; the delete is explicit, auditable and testable — metadata-db.md documents the choice), then the version row; when it was the sole version the entity row goes too (the entity purge, §3.2).

Before every destructive write, both pipeline draft purge services ask the pin question: live exact parent/dashboard pins for a nonsole draft; the existing entity guard for a sole draft (live exact parents, stored any-version dashboard pins). A blocker refuses 409 pipeline.version.pinned, naming the parent pipelines and dashboards, and preserves the draft hash, executions and pointer. The hash-guarded path checks the supplied hash first; a stale hash retains pipeline.version.conflict. The POST repeats this check even if a dialog GET found no blockers. The concurrent pin/write window remains as stated in §3.5.3.

DELETE FROM pipeline_executions
 WHERE (pipeline_id, version) = (:id, :draftVersion);

DELETE FROM pipeline_versions
 WHERE pipeline_id = :id AND version = :draftVersion
   AND status = 'DRAFT' AND body_hash = :expectedHash

Redis result keys are not deleted — they expire on their own TTL, and a purge leaves zero orphan execution ROWS (asserted by test); an expired key is not a reference. If the purged draft had become current_version (the development fallback of §3.4), the same transaction recomputes the pointer. 0 rows on the version delete ⇒ 409, as before.

5.5 Drafts are a deployment capability (039)

D7 says "receiver environments never author" — nothing enforced it. Now something does, mechanically, as one block of deployment-role settings (configuration.md §3.19; the codebase groups config one-per-concern — auth, executor, staging — so deployment is the house shape):

datapipelines:
  deployment:
    name: dev                      # LABEL ONLY — nothing branches on it (below)
    authoring-enabled: true        # the capability this section enforces
    promotion:                     # RESERVED — not implemented as of this round (§10.6)
      server-key: ...              # INBOUND: what I accept (receiver role)
      target:                      # OUTBOUND: where I push (sender role)
        base-url: ...
        server-key: ...

• deployment.authoring-enabled, default true — named for the CAPABILITY, not the environment (the boot refusal covers PIPELINE, TEMPLATE and — since #194 lane D, C14 — parameter-set drafts: AuthoringStartupCheck refuses startup on an authoring-disabled deployment holding any of the three): someone runs this on ONE box, authors there and runs there, and that server is "production" in every ordinary sense, so a key that gated authoring on an environment NAME would lock them out of the only server they have. A promotion receiver turns it off; everyone else is on by default.
• deployment.name is a LABEL, and nothing may branch on it. This is the rule that makes the bundling safe rather than a regression — putting the label next to the capability re-creates exactly the temptation the capability naming exists to avoid. Its one consumer is the startup posture log line (name + authoring state); it is not on /info (permitAll), and a guard test pins that no production code reads it in a conditional — stronger, that no production source names the key at all beyond its log consumer.
• Enforced at the write path, fail-closed (C3), with the catalogued refusals pipeline.authoring.disabled / template.authoring.disabled (§13.13/§13.9, 403) naming the reason. Every authoring write refuses on a receiver: pipeline/template create, update/draft, release, discard and delete — REST and MCP alike, at the shared services. Reads, execution and import are unaffected.
• Draft creation hangs off the WRITE path only — never off "a released row was written". This is the trap this section exists for: promotion imports RELEASED versions into a receiver, and any draft-creation logic attached to the released-row path would fire on every import and create drafts on exactly the deployments this rule forbids them on. The import paths (§9.2) insert RELEASED rows directly and never open drafts; the round's tests assert it.
• Boot checks (configuration.md §7): a deployment with a promotion server-key configured — meaning it RECEIVES — AND authoring enabled WARNs loudly (D7's violation stated in config) but does not fail (a one-box deployment may legitimately be both). This check is currently one-sided: the promotion sub-block is reserved for the round that implements it (§10.6's fenced sample), so the authoring half ships now and the promotion half slots into the check's seam without rework. Authoring disabled while DRAFT rows exist REFUSES startup, naming them: someone authored on a receiver and version alignment may already be broken (§9.3) — better found at boot than at the next promotion's 409.

6. Templates: Same Lifecycle, Plus the Pin Rule

Parameter-set create, draft write and validated import (including promotion receive) recheck template pin existence, live status, type and dialect after acquiring the workspace writer guard. Customer selector probes remain outside the metadata transaction. A purge or discard that wins before the guard makes the set writer refuse without storing a pin.

template_versions gains the identical statuses, hash column, one-draft index, and four write paths (mirrored codes template.version.conflict, template.version.not_draft).

The template metadata asymmetry (v1.3). §3.7's metadata-rides-the-release is a PIPELINE rule, and it works there because the pipeline body carries its metadata — the draft row stages it inside body_json. A template's display_name/description live on the index row templates only and are not part of the versioned artifact, so a template draft stages what CAN be staged (engine/dialect/is_library/imports/body — exactly the canonical body of §4.1) and the index metadata keeps moving at save time. Templates have no rename (name is the identity), so §3.7's draft-write-time name check is pipeline-only. Template draft purge is likewise simpler: nothing references a template_versions row by FK (pipeline pins are numbers in JSON), so a template draft is always hard-deleted with no execution cleanup to do. A RELEASED template version is discarded/restored exactly like a pipeline release (§3.5's table, template code spellings), and template.in_use — the pin guard that already guarded entity delete — is the code for a pinned version's discard/purge too.

Because drafts pre-allocate real version numbers, a DRAFT pipeline can pin a DRAFT template version by its number — this is the intended editor loop (edit SQL → run the node → tweak → release), and it is why drafts live in the version tables (D1). The rule that makes it safe:

A pipeline may be released only when every template version its body pins is RELEASED. Draft pins are legal while the pipeline itself is a draft.

Release ordering is therefore always templates-first. Pins are immutable references, so no cycle is possible.

7. REST Surface Changes

Additive; existing routes keep their shapes. Exact wire contracts land in rest-api.md at implementation.

The 101 verbs are human verbs (D4 family). All carry MUTATE_PIPELINES_TEMPLATES and are session-only — an API key, however scoped, is refused (auth.session.required); there is deliberately no MCP tool for any of them (§12.2's rule, applied to purge/discard/restore/switch exactly as to release). The one non-session exception is POST …/current for an operator rolling a receiver forward or back — still session-only on REST; receivers are driven from the UI. Every verb emits its audit event (§3.5 of enums.md §15): pipeline.version.discarded / restored / purged, pipeline.purged, pipeline.current_switched, the template twins, and — since #332 — the same five for the parameter-set, visualization and dashboard families (their release events too, each cascade named cascade_from_<family>_id, the pipelines mould's auditRelease twin).

7.1 Authoring reads return the working version (039)

The principle: a RELEASED version is never modified, and if a draft exists it is reused, no matter what. Authoring reads — REST GET /pipelines/{id} and GET /templates?name=..., the MCP pipelines_get / templates_get tools, and the editors — therefore return the working version: the draft if one exists, else current_version. They always state which version and which status they returned; the caller never infers it. Before 039 the REST and MCP default was the released body with a draft pointer beside it — an agent that read released while a draft was open would rebase its edit on stale content and quietly discard the draft with its next write.

Three pins:

• current_version is NOT repointed at the draft. It means latest released, and execute-default, the datasource reverse-scan join and the editor's fallback all read it that way — repoint it and executions run unreleased code. The working version is DERIVED (draft-exists ? draft : current), never stored; there is no schema behind this section.
• Explicit reads still win. An explicit version argument (or versioned URL) returns exactly that version; this changes only the DEFAULT, and only for the authoring surfaces. Execution keeps reading current_version. Superseded by D56 (§7.2): execution's default is the working version too.
• The no-op rule (§5.1) is what makes this safe to act on: a draft exists iff the content genuinely differs, so "the draft is the working version" is never a phantom an unchanged save left behind.

Exact spellings (v1.3, fixed in rest-api.md): the hash precondition travels as the standard If-Match request header carrying the body_hash — on PUT, release and discard, for pipelines and templates alike; absent/blank ⇒ 400 pipeline.execution.invalid_parameter_type with details.reason = "precondition_missing" (a caller that did not participate in the protocol at all, not a conflict). Release and discard distinguish their refusals: no DRAFT exists ⇒ pipeline.version.not_draft / template.version.not_draft (409); a DRAFT exists but the hash is stale ⇒ *.version.conflict with the current state in details. The executions surfaces carry §8's marker as a draft_run: true|false field on each execution record.

MCP authoring (the pipelines_update tool, and templates_update (117) for templates — the same TemplateDraftService.write call PUT /templates makes, with the If-Match hash as the required expected_hash argument) calls the same PUT and therefore lands its work as drafts on existing entities. An agent iterating produces one draft row that it keeps overwriting — the pile the old PUT-per-save semantics created is gone. A human releases from the UI when satisfied (D4). The pipelines list gains a "drafts pending release" badge so unreviewed agent work is visible.

7.2 Execute with no version runs the working version (D56, 099)

The default everywhere is the working version: the draft when one exists, else the latest release. It is the rule §7.1 already applied to authoring reads, now applied to running as well, and it is what "we always run the LAST version" means:

One place resolves it: PipelineService.workingVersion(workspaceId, record). It was three inline copies (the REST execute controller, PipelineExecuteTool, pipelines_get) before 099, which is exactly the drift the D6 refactor removed from the LOOKUP and left in the DEFAULT.

Posture. Under authoring-enabled=false (the hardened default, configuration.md §244) no draft can be created at all — create, update, release and discard are all refused with pipeline.authoring.disabled (§5.5) — so on a non-development server the working version is a RELEASED version by construction, and no configuration says otherwise. That is the ruling's second half ("on servers which are not development, we will always have RELEASED pipelines, no DRAFT at all") held up by a mechanism rather than by a promise.

Two consequences worth stating:

• A draft run is marked. The execution record already pins the version that ran; the executions list and detail render DRAFT beside a draft run (§8, ui-screens §4.8) so a reader is never guessing whether a result came from reviewed content.
• There is one "nothing to run" case at the DEFAULT, and it is not a fresh pipeline: creation always writes version 1, so the only way to a version-less pipeline is purging the sole draft of a never-released one (§3.2 — the purge deletes the entity with it). Execute answers 404 pipeline.execution.not_found there — the same refusal an out-of-range explicit version gets, with details.pipeline_version naming it. A pipeline whose every release is DISCARDED (entity DISCARDED, pointer NULL) answers the same refusal at execute-default, and an explicit execution of a DISCARDED version is refused the same way — a retired version is not executable; restore it first (§3.1).

8. Executing Drafts

Drafts are executable — the editor's test loop depends on it (single-node run, full-draft run). Because a draft is a version row, the existing composite FK records the run against the real draft version number; nothing about the execution schema weakens.

• A draft run is DERIVED, not recorded (ratified 2026-08-31). For an execution of version N: it was a draft run when started_at < released_at, or when that version has no released_at (still DRAFT). No schema column. The derivation is sound because a version's lifecycle is one-way — RELEASED never returns to DRAFT (§3.1: a write after release opens a NEW version), numbers are never reused (§3.3), and 101's restore returns a DISCARDED version to RELEASED keeping its original released_at — so each (pipeline_id, version) has exactly one released_at to compare against, forever.

  The precondition, stated because it is not obvious. started_at is application-supplied, not a database default: ExecutionRepository binds :startedAt from the record (its INSERT names started_at explicitly). So this comparison spans two clocks whenever the release and the execution are served by different instances. The failure window is sub-second and either side of a release; the consequence is a history LABEL, never execution behaviour, never promotion eligibility (§10.3 reads status, not this derivation). released_at must therefore be set by the database (NOW()) at release, so at most one of the two timestamps can drift. If a future requirement makes the draft/released distinction load-bearing rather than informational, record it instead of deriving it.

• Draft executions appear in history with a draft marker; they are never promotable and never count as validation for release (the human decides that).

9. Cross-Environment Identity and Import

9.1 Why version numbers must be global identities (verified defect)

Both import paths renumber today: pipeline import strips the version field (PipelineImportService SERVER_FIELDS) and allocates current_version + 1; template import calls the same bump-and-append. That breaks the export bundle round-trip silently: a dev pipeline pins acme/lib/foo.sql@2, the bundled template lands on a fresh target as acme/lib/foo.sql@1, and the pipeline's pin fails template_version_not_found. The export/import pair only round-trips when both environments' numbers happen to be in lockstep — i.e., never after any drift. Pins are version numbers; cross-env renumbering is a correctness-breaking hazard, not a cosmetic one. D5 fixes this by construction.

9.2 Import with preserved versions

The import payload may carry version (promotion always sends it). When present it is honored; when absent, today's allocate-next-local behavior applies. Target-side rules:

The pointer rule on import (D60, 101). An import never moves an existing pointer: promoting v5 onto a receiver whose pointer names v3 inserts the version and leaves the pointer at v3 — the human switches when they are ready (POST …/current, §7). The single exception is the FIRST import of an entity that has no current at all (a never-released entity, or one whose every release was discarded): nothing exists to break, so the pointer is set to the imported version. Index metadata (name/display_name/ description) rides the pointer, not the number: it adopts the imported body's values exactly when the import moved the pointer. Version-less imports (no version in the payload) allocate max(version) + 1 — never current_version + 1 (the pointer can be NULL or stale) — and follow the same pointer rule. Gaps in the number sequence are expected and harmless.

Hash recompute guard: the target recomputes the body hash from the payload body and refuses with pipeline.import.hash_mismatch if it differs from the declared hash — this catches transfer corruption and canonicalization drift between app versions in one place.

Wire spelling (v1.3): a payload carrying version must also carry body_hash (absent ⇒ hash_mismatch with details.reason = "body_hash_missing"); released_at is honored when present (export emits it) so §8's derivation stays truthful on the target. The stored released_by is the IMPORTING actor — the source's releasing user may not exist locally (the promotion case is §10.6's service principal, out of this round). The template import mirrors the table entry-for-entry with template.version.conflict; the template-side hash-mismatch code does not exist in §13 and this round adds no rows, so the internally-inconsistent entry surfaces as template.version.conflict with details.reason = "hash_mismatch" — a catalog gap raised to the operator, not papered over.

9.3 The price of number preservation, accepted

If a receiver ever authors locally (a process violation, D7), its local version numbers collide with future dev releases. The outcome is a loud 409, resolved procedurally (re-do the change in dev, release, promote). The design fails safe — it never silently diverges, and with receivers that never author, the price is never paid.

10. Promotion (UI-Driven, Separate Use Case)

Against exactly one configured higher environment (base URL + the pre-shared server key of §10.6; the keys themselves are defined in configuration.md §3.19, not restated here, per that doc's sole-authority rule). Shipped by 055; the receiver's two endpoints are specified in REST API §18.

10.1 Policy

• No hotfixes (D7): a bug fix is a new release in dev, promoted like any change. Receivers never author; the sole writer of a receiver is promotion.
• Promotion is triggered by a human from the UI (D8). No MCP tool. No schedule, no auto-promotion.

10.2 The listing rule (what the UI shows)

The promotion screen lists only pipelines that are RELEASED and whose current version number is greater than the target's current version for that pipeline (a pipeline the target does not have counts as target version 0). Drafts are never listed. The same holds when the pointer itself names a draft — §3.4's development-posture fallback or a manual switch (#563): an object whose current version is a DRAFT is not promotable and is invisible to a promoter whatever releases it holds; it returns when the pointer names a release. Same-version entries are never listed.

The wire inventory (REST API §18.1) obeys the same released-pointer rule for all five families, omitting draft-pointer and never-released objects (#575).

The same rule is the promoter LENS (178, Auth §11A.1). Since R2 a promoter sees, on EVERY read surface — lists, levels, counts, search, gets, the editor source, the MCP tools and resources — exactly the set this screen lists, for pipelines AND templates: the rule is computed once per request (PromotableView) and the page and the lens read the same object, so they cannot disagree. The template arm, stated for the first time here (the push closure only implied it): a template is promotable iff it has a current RELEASED version and the target lacks its id, or holds a different hash at a lower version — the four lines above with the template's id for its name. A hidden object answers a promoter exactly as an absent one, and a visible object's pending DRAFT is invisible as well — the working version a promoter reads (§7) is the current release, never the draft — on every read that resolves version content, the node SQL, the checks and the used-by pins included (178b); when the target cannot be read the lens shows NOTHING (fail closed) and the screens say why.

The delta is computed from the target's inventory: the promotion orchestrator reads the target's per-pipeline and per-template (name, current_version, body_hash) (REST API §18.1) and compares. By name, not by id (the shipped correction to this paragraph's first draft): workspace and template names are a global namespace and pipeline ids are globally unique but locally opaque, so a name is the one identifier that means the same thing on both deployments. The comparison is:

• Same body_hash ⇒ nothing to push (version for humans, hash for machines).
• Different hash or absent ⇒ the pipeline is promotable; only its latest RELEASED body is pushed (D6) — intermediate released versions are not transferred; the target executes current_version only and its own history references its own numbers.

10.3 Server-side guards (independent of the UI)

The UI rule is convenience; the server enforces the same constraints regardless of caller:

1. The promotion orchestrator refuses to push a pipeline whose candidate version is not RELEASED → pipeline.promotion.not_released.
2. It refuses to push a version not greater than the target's current version for that pipeline → pipeline.promotion.not_newer. Same-version pushes are a bug, not a no-op to swallow.
3. The receiver enforces §9.2's import table — conflict, idempotency, hash recompute.

Both ends guard; a client bug cannot smuggle a draft or a stale version through.

10.4 Push order (dependency closure)

Per promotion batch, push in topological order:

1. Template versions referenced by any pushed pipeline (direct node refs plus the transitive imports_json closure — the export bundle already computes this set), plus — since #10 L1c — the transform pins of any pushed visualization.
2. Child pipelines referenced by PIPELINE nodes (recursively — the export bundle does NOT include these today; promotion computes the closure itself).
3. The pipelines, children before parents.

The receiver applies the batch in ONE transaction in that order, then — since #10 L1c — the parameter sets' entries and, LAST (D61), the batch's visualizations and dashboards: the transfer families bind through the families' strict readers and land inside the transaction, where the just-landed templates and pipelines are what the import lens resolves against. A refusal at any entry rolls the whole batch back — the templates included. Since #517 the batch's transaction acquires the workspace writer guard FIRST (§5.3), so no release, purge or discard in the receiving workspace interleaves with a batch; a wait past lock-wait-seconds refuses the batch with pipeline.release.busy and nothing lands.

The transfer arms carry one family each (the L1c-b round, O1): a dashboard root's pinned visualizations travel as entries of the visualizations arm — deduplicated by name + version

• body hash against the explicit visualization roots — and the dashboards arm carries dashboards alone, because each arm's entries are bound by that family's reader. Both arms are count-bounded before the transaction opens (O7): at most datapipelines.visualization.max-visualizations-per-dashboard entries, refused body_invalid with reason: too_many. The ceiling is AGGREGATE (the L1c-c round): the value also caps each whole arm as an ADDITIONAL ceiling on top of the per-document bound the key's name describes — two dashboards that are each individually valid can together exceed it, and such a batch refuses WHOLE (no partial writes, no automatic split); a genuine batch at exactly the configured ceiling lands whole. A landing is judged by the RELEASE rules (O2): a batch entry lands RELEASED, so a pin it carries that exists on the receiver but is not RELEASED refuses with the family's release code (visualization.release.dependency_not_released / dashboard.release.dependency_not_released); a pin the receiver lacks at all stays the import lens' precise import.missing_dependency. The refusal at ANY entry rolls the whole batch back — witnessed end to end by the transfer E2E's refused-dashboard receive case (the L1c-c round): earlier batch members land, the dashboard refuses on landing, and no template, visualization, version, current pointer or accepted-audit row survives.

Pins are immutable and cycle-free, so the order always exists. Templates/pipelines already present at the same version and hash are skipped (idempotent).

The closure is read at the exact PINNED versions, without the promoter lens (the owner's ruling of 2026-10-02, #344; Auth §11A.1's lens clause). §10.3's guards and the lens govern the ROOTS a human selects — a root the lens hides refuses with its family's 404 — never the dependencies they pin: a pinned template §10.2's view hides (already on the target, or behind it) still rides, and the skip rule above is the only thing that leaves one out. The export bundles (rest-api §5.9 and the families' export routes) follow the same rule.

10.5 Datasource pre-validation

Pipelines reference datasources by name. Before pushing anything, the orchestrator collects every datasource name the batch references and verifies each exists on the target; a missing name fails the whole batch with one consolidated error (mirroring the import service's combined missing_datasources report) rather than failing mid-batch.

A promoted pipeline's release checks travel in its body (140: checks[] is body content, versioned like nodes). The receiver re-runs them on its OWN datasources at its release step — the same §5.3 gate a human release faces, with no override channel: fail or error refuses the batch with pipeline.check.failed. A receiver without a check's datasource records error, which is the truth — the alternative is importing a number nobody verified.

10.6 The promotion-peer credential — a shared server key (ratified 2026-09-01)

§10.1 said "base URL + API key", which was under-specified: an API key here is workspace-pinned and user-owned, and neither property is right for one deployment writing to another.

Ratified shape (operator, 2026-09-01): a pre-shared server key, not a principal. Shipped 2026-09-02 (055). The RECEIVER holds a long secret in configuration; the SENDER holds that same secret alongside the target's base URL and presents it on the promotion call; the receiver validates it before accepting any payload. No service account, no users row for the credential itself, no scope matrix entry — promotion is a deployment trusting a deployment, and modelling it as a principal was more machinery than the problem needs.

The configuration shape. It lives inside the datapipelines.deployment block (§5.5's amendment — the deployment-role settings are grouped, and the promotion sub-block mirrors the inbound/outbound split this section draws). It is now shipped and documented in configuration.md §3.19, which is the operator-facing authority; the sample below is retained as this section's own illustration:

# receiver (e.g. uat) — absent means promotion is refused
datapipelines:
  deployment:
    name: uat
    authoring-enabled: false           # receivers never author (D7, §5.5)
    promotion:
      server-key: ${DATAPIPELINES_DEPLOYMENT_PROMOTION_SERVER_KEY:}

# sender (e.g. dev) — the target and the same secret
datapipelines:
  deployment:
    name: dev
    promotion:
      target:
        base-url: ${DATAPIPELINES_DEPLOYMENT_PROMOTION_TARGET_URL:}
        server-key: ${DATAPIPELINES_DEPLOYMENT_PROMOTION_TARGET_KEY:}

The implementing round adds the block to configuration.md in the commit that makes it real.

The gap this shape did not close, and how it was closed (R7, ratified 2026-09-02). pipeline_versions.created_by and pipeline_executions.triggered_by are NOT NULL REFERENCES users(id). A payload authenticated by a server key carries no user, and the source deployment's user ids are meaningless on the receiver — different users table. So an imported row still needs an actor that exists locally.

Three options were weighed: a reserved non-interactive users row; making the column nullable with a CHECK; and mapping by email (rejected outright — it silently creates or mis-attributes when the human has no account on the receiver).

The ruling took the first option's shape but not its mechanism: a system service account provisioned at first boot, not created by a migration, and used for promotion AND for every other non-user-bound write the system makes now or later — the retention job, the stale-execution sweeper, anything automated that comes after. It is provisioned through the same createUser path the bootstrap admin uses, with the same create-if-absent and insert-race semantics, and read back through ONE well-known lookup so nobody mints a second actor. Login is disabled by construction: its provider name is reserved at startup validation, its address is under RFC 2606's unresolvable .invalid TLD, and the local-password paths refuse it. The full contract is Auth §4.5.

The FKs therefore stay NOT NULL and no schema change was needed. Every promoted row is stamped with that actor, and the receiver's audit event carries the provenance the row itself cannot: source_env (the sender's deployment.name) and a truncated fingerprint of the key that authorised the push — never the key. The releasing human stays recoverable from the artifact's own released_by on the source deployment.

The machine-auth note's F10 defers to this section; F2's service-account question is answered only for the ACTOR half — an application EXECUTING a pipeline is a different problem from a deployment PROMOTING one, and D16's own headline is that execution is blocked by the SSE contract before auth is even reached (R9).

11. Schema Changes (Amendment for Implementation)

metadata-db.md remains the sole DDL authority (its rule D4); these land there at implementation time, in the same commit as the repositories:

pipeline_versions and template_versions gain:

Plus the two one-draft partial unique indexes (§3.3). No column is added to pipeline_executions — §8 derives the draft/released distinction from released_at, which is why that column is database-generated (NOW() in the release statement, not DEFAULT — it is NULL until the first release of that version) rather than application-supplied. The migration is V6__version_lifecycle.sql; metadata-db §4.5/§4.9 carry the DDL (the authority) and §5A carries the promotion classification registry the migration test parses.

The immutability KDoc blocks on both repositories are rewritten to the §3.1 discipline.

12. The Agent-Facing Skill Must Land With the Code

modules/mcp-server/src/main/resources/skill/core.md (served as /skill.md, #242) is what an agent reads before it touches this product. It is not documentation about the system — it is the system's instructions to its own callers, and MCP agents are the primary authoring surface here (contract §2; the editor is a viewer/executor). A lifecycle change that reaches the API without reaching the skill leaves every agent operating on the old contract.

This is not hypothetical for this spec. SKILL.md:51 currently states:

Versioning — every save creates a new version.

D2 makes that false. After this round a save on a released version creates a DRAFT, and a save on a draft overwrites it in place. An agent holding the current skill will believe its pipelines_update published something, and it did not.

The rule: the skill is updated in the SAME commit as the behaviour it describes — the same discipline §13 applies to catalogue rows and their constants, for the same reason. A skill that lags is worse than one that is silent, because agents act on it with confidence.

Audited 2026-08-31 at 7f87118: the skill is otherwise accurate, so :51 is the only lie this spec introduces. All 18 McpToolCatalog.NAMES tools are named in it, the "18 MCP tools" count matches, and it already discloses the REST/MCP execute asymmetry the architecture audit records as D6 (SKILL.md:146 — "REST execute accepts Idempotency-Key; the MCP tool has no key"). Do not update the skill for this spec before the behaviour ships — a skill describing an unimplemented lifecycle is the same defect in the opposite direction, and the architecture audit's M11 exists because three such claims are live in deployment.md today.

12.1 What this spec obliges the implementor to change

12.2 What the implementor must decide, not assume

The skill says agents author and the UI releases. Whether pipelines_update returning a draft is surfaced as a distinct tool result, or as the same result shape with a status field, changes what the agent can reason about — and the MCP tool surface is mcp-server.md's authority, guarded by McpToolSurfaceSpecDriftTest and the McpToolCatalog binding (033). If the round adds or reshapes a tool, that spec, the catalogue and the skill move together; if it only changes a payload, say so explicitly in the handback so the drift guards are known to have been considered rather than missed.

No new MCP tool for release — D4 and §14 both settle that. The absence is deliberate and the skill should state it, so a later reader does not read the gap as an oversight and "fix" it.

12.3 The §12.2 decision, made and recorded (v1.3 / 035)

The draft-returning pipelines_update is a payload change, not a surface change: all 18 tool names stand, no tool is added or removed, and mcp-server.md §6.1 is untouched. Concretely: pipelines_update gains the REQUIRED expected_hash argument (same input as pipelines_create plus id and expected_hash), and both create and update results gain status, body_hash, current_version and — when a draft exists — a draft pointer; pipelines_get merges the same fields into the returned body. §6.2.5's prose and §6.2.2's result note in mcp-server.md were amended to say exactly this (the tool is documented in prose, so McpToolSurfaceSpecDriftTest's schema comparison is unaffected by construction). Making expected_hash required is deliberate: D3 says no last-write-wins EVER, and an optional precondition is a protocol hole an agent will fall through — the refusal for omitting it is the input schema itself.

12.4 The general rule, beyond this spec

Any round that changes what an agent can do, must do, or will be refused for, updates SKILL.md in the same commit. The test is not "did the API change" but "would an agent holding the current skill now be wrong?" If yes, the skill is part of the change set. The skill has no drift guard today — nothing fails when it goes stale — so this rule is carried by review, and a round that touched agent-visible behaviour without touching the skill should be asked why in its handback.

13. Testing Requirements

House discipline: every guard below must be falsifiable — revert the production change and the test goes red (a guard that cannot fail is not a guard).

• Write-path unit/integration: draft create (copy-on-write honored, released row untouched), draft write (in-place, no new row), release (pointer bumps, statuses flip), discard (delete vs DISCARDED-by-FK). Two concurrent draft creates ⇒ exactly one winner, one conflict.
• Hash precondition: every mutation with a stale hash ⇒ 409 with current hash/author details; with the correct hash ⇒ succeeds. Falsification: remove the WHERE body_hash predicate and watch these go red.
• One-draft index: second draft insert violates the partial unique index.
• Release pin rule: pipeline release with a DRAFT template pin ⇒ pipeline.release.dependency_not_released; with all pins released ⇒ succeeds.
• Import table (§9.2): each row of the conflict table as a case; hash recompute guard catches a doctored payload; the D60 pointer rule (subsequent import never moves the pointer; first import onto a current-less entity sets it).
• Promotion guards: draft selected ⇒ pipeline.promotion.not_released; same-version ⇒ pipeline.promotion.not_newer; push order respects the closure (child pipeline absent on target ⇒ parent not pushed); datasource pre-validation fails batch atomically.
• The lifecycle model test (101's acceptance gate). VersionLifecycleModelTest generates seeded random event sequences (length 1–25) over real Postgres and the real services, in both postures, and after EVERY event asserts the invariants — ≥ 1 version row; ≤ 1 DRAFT; the DRAFT is the highest number; numbers strictly increase; the pointer is live-or-NULL and eligible for the posture; after a pointer move it is the highest eligible; no exact pin from a live version points at a DISCARDED/purged version; the entity status equals the derivation; names unique; a purged version leaves zero execution rows — AND that the event's outcome equals §3.5's row for that shape. The sequences replay §3.5's own rows (parsed, never copied) before the random phase. The gate runs 2,000 sequences per posture with a bounded, measured runtime; the seed prints on failure with a shrunk counterexample. Falsify the model itself — comment out one service rule (e.g. eligibility under hardened) and the property must fail within the run; a model test that cannot go red is a rubber stamp.
• Drift: new §13.13 codes ↔ PipelineErrorCodes constants. PipelineErrorCodesSpecDriftTest parses the catalogue generically and asserts BOTH directions, with a per-domain non-vacuity guard — so this is enforced automatically. The operational consequence is that the §13.13 rows and the constants must land in the SAME commit: a catalogue row without its constant turns main red immediately, and the drift test lives there permanently. The same applies to the mirrored template.* codes and TemplateErrorCodesSpecDriftTest.

14. Out of Scope (Deliberate)

• MCP release tool — decided against (D4). The REST release endpoint exists for the UI; an agent holding a raw MUTATE-scoped API key could call it directly, which is acceptable — "not in the agent" means no first-class tool surface. The 101 verbs (purge, discard, restore, entity purge, switch) extend the same ruling: no MCP tools, and REST carries them session-only.
• UI for the lifecycle verbs — 102 puts buttons on them; this round ships services, REST, the spec and the proof.
• Bulk "release all drafts" — per-pipeline release only for now; a bulk action can be added to the promotion screen later without schema change.
• Multi-hop promotion, promotion scheduling, promotion history UI — the single-target, human-triggered flow ships first.
• Merge/rebase tooling for conflicting drafts — the conflict response gives the current hash; rebase is manual by design at this scale.
• Discarded-release retention jobs — DISCARDED release rows accumulate only when a human discards one; a retention job is deferred until measured. (The executed-draft tombstones this used to describe are gone — drafts are purged with their executions, §3.1.)

15. Ratified Decisions and Remaining Open Items

Items 1–3 were ratified by the operator on 2026-08-31 and are written into their sections; they are recorded here so a reader sees what was decided and why, without re-opening it.

Remaining open, deliberately:

1. The receiver inventory API shape. §10.2 needs the target's per-pipeline (id, current_version, body_hash). rest-api.md owns the endpoint's shape at implementation; this doc owns only what promotion needs from it.
2. Editor UX for the release action — button placement, the "draft pending release" badge, the conflict-reload flow. pipeline-editor.md's next revision, not this doc's.
3. Whether the §10.6 service principal later serves the application-execution case the machine-auth note describes. That note settles it once ratified; §10.6 constrains only promotion.

Change Log