Home / Projects / Orchestrata Ultimate / Decisions / ADR index / ADR-0019
ADR-0019: Gate reads require causal consistency — snapshot + watermark, not replicas
| field | value |
|---|---|
| Status | Accepted |
| Date | 2026-07-24 |
| Register | factory (docs/adr, 4-digit ids) |
| Source | docs/adr/0019-gate-reads-require-causal-consistency.md — permalink at bdafa201a (unchanged on master since that commit) |
| Wiki register entry | ADR-0019 (wiki status: Accepted) |
The record
- Status: Accepted
- Date: 2026-07-24
- Spec: SPEC-033 §3, §4
Context
Factory gates fail closed (ADR-0002), which makes the fact store a liveness SPOF: one transient blip parks every worktree. The proposed mitigation was to serve gate reads from a read replica.
Two defects:
- It silently inverts the failure posture. A replica whose state predates the commit's fact promotion evaluates the rules against old facts. A real violation is then invisible — no fault, no park, just a stale answer. Fail-closed degrades to fail-stale-pass, which is worse than fail-open because it is silent.
- The mechanism does not exist on the chosen datastore. ADR-0003 selects Neo4j, and the repo runs
neo4j:5Community (CI workflow anddocker/docker-compose.yml). Clustering and read replicas are Enterprise-only. §3's liveness mechanism was unavailable on §4's engine — an internal inconsistency at design level, not a build gap.
Decision
- Gate reads must be causally consistent with the commit under evaluation. The gate refuses to evaluate until the read view's watermark ≥ the commit's fact-write (Neo4j bookmarks, or an explicit commit watermark over a read-through snapshot).
- Implement as snapshot + watermark, not replicas — this works on Community and does not depend on an Enterprise licence.
- With the watermark in place, a stale view parks and retries (a transient fault) rather than evaluating; availability is bought only where reads are already fresh, preserving fail-closed.
- If Enterprise clustering is ever adopted, that is a one-way door requiring its own ADR.
Alternatives rejected
- Replica reads without a freshness condition — the fail-stale-pass inversion above.
- Always read the primary — correct but concedes the liveness SPOF entirely; the watermark buys back availability wherever the view happens to be current.
- Adopt Neo4j Enterprise now — a licensing/one-way-door decision taken for a mitigation that snapshot+watermark provides without it.
Consequences
- Adds a freshness precondition to every gate evaluation, and a park-retry path distinct from escalation.
- Ties gate liveness to promotion latency: slow fact promotion directly becomes gate wait time.
- Removes the Community/Enterprise contradiction from the spec.
Mechanism change 2026-07-26 — bookmarks are not available on the new engine
The decision above sanctions two mechanisms: "Neo4j bookmarks, or an explicit commit watermark over
a read-through snapshot". The build implemented the first. The Neo4j -> orchgraph cutover
(plans/neo4j-to-orchgraph-migration-plan.md) removed it: the orchgraph engine returns no Bolt
bookmarks (session.last_bookmarks() is empty) and silently ignores any bookmarks supplied to a
session.
That did not merely disable the guarantee, it inverted it. await_watermark's probe was "open a
bookmarked session, run RETURN 1" — a freshness check only on an engine that enforces bookmarks. On
orchgraph it succeeds immediately for every token, including one no write ever minted, so the
fail-closed gate had become fail-open while still reporting success. Measured, not inferred: the
regression test asserting WatermarkTimeout on an unreachable token failed against the new engine.
The second sanctioned mechanism is now the implemented one. Every write_fact/promote mints an
opaque token and stamps it on the :FactNamespace control node inside the same transaction as the
facts, so a reader that can see the token is guaranteed to see the facts. await_watermark polls until
that token is visible in the read view. This is engine-independent, strictly stronger than the previous
probe (it verifies data visibility, not merely that a session opened), and fail-closed by construction:
an unknown watermark is never visible, so it can only ever time out.
PromotionRecord.bookmark is renamed watermark to name what it now carries.
No part of the Decision changes — only which of its two permitted mechanisms is built.
Provenance
Round-3 Opus (#8), including the code verification that the repo runs Neo4j Community while §3 assumed replicas. The park-with-retry-vs-escalate distinction originates in the same review's liveness critique. The 2026-07-26 mechanism change is recorded above.
Related ADRs
Rulings cited
None.