Home / Projects / Orchestrata Ultimate / Decisions / ADR index / ADR-0076

ADR-0076: The semantic/RAG ingest leaves the prod promote; readers check freshness at runtime and trigger refresh on demand

← ADR-0075 · ADR-0077 →

The record

Status: Proposed (recommendation: Accept) · Date: 2026-10-07

Context

ADR-0074 (Accepted 2026-10-07) kept the Memgraph ingest inside the promote's graphs success gate — its rejected "timer re-embeds later" alternative argued that decoupling loses the guarantee that the semantic graph matches the deployed tree. Mohamed ruled the opposite direction on 2026-10-07: promotion ships code, not data — "no separate code promotion … it works in production from data needs refresh, if data needs refresh, when code runs it checks for freshness and triggers refresh if needed." Scope ruled: semantic/RAG only (the memgraph chunk/embedding ingest); the operational-KG chain (schema, AST full sync, architecture sync, spec enrichment) stays in the promote — it is local, unmetered, and the code-intelligence tools read it.

The measured facts that make the ruling economic are ADR-0074's: the ingester re-embeds all 5,740 chunks on every promote (570 paid OpenRouter calls, 5 min 20 s of the stage), on text that did not change, and a provider lapse failed today's promote AFTER verify had passed, forcing a restart-and-rollback cycle. The semantic readers (semantic_search.py, the context-pack retriever, wiki-rag) consume the graph at runtime; they are the natural owners of freshness.

Decision (proposed)

1. The promote stops deriving semantic data. bootstrap_graphs.py --all becomes --kg in the promote's graphs stage (the seam already exists; ADR-0074 measured it). The Memgraph step (ingest_orchestrata_ultimate.py) is no longer a promote stage, so no promote pays for, waits on, or fails on embeddings. "Promoted to prod" means: the deployed commit plus the operational KG derived from it.
2. A successful promote triggers the refresh. When the promote completes (tree moved, verify green), the promoter triggers the embedding refresh — the promote neither waits on it nor gates its success on it, but the refresh is the deterministic consequence of shipping code, not a reader's accident. A failed or deferred refresh alerts and is retried by the periodic safety net.
3. The refresh is single-flight and incremental. The trigger starts an on-demand refresh unit (plus a periodic safety net) that runs ADR-0074's item-1 engine — embed only chunks whose id is new or whose content_sha256 changed, delete chunks whose id left the tree. A refresh in progress is observed by concurrent triggers and readers, not restarted.
4. The provider probe re-homes to the trigger. Before starting, the refresh checks the embedding provider's balance (ADR-0074 item 2's probe); a 401/402/403 or an unaffordable estimate defers the refresh and alerts — it never fails a reader and never fails a promote.
5. Read-time freshness is the backstop, and staleness is never hidden. The ingest records the source commit id it derived from (a marker property on the graph it writes). Every semantic reader compares that marker with the deployed /opt/orchestrata commit before answering; if the graph is stale (a triggered refresh failed or has not landed), the reader triggers the refresh and surfaces the staleness loudly — it does not answer from stale data as if it were fresh. Solving the problem means the refresh happens; the check ensures a silently stale graph cannot persist.
6. ADR-0074 item 3 is superseded; its items 1–2 survive, re-homed from the promote to the refresh path. On Accepted, ADR-0074's file gains a supersession note to that effect.

Alternatives rejected

• Keep the ingest in the promote (ADR-0074 as accepted): the ruling's direction is the opposite; re-argued here only to record that the rejected alternative's cost — a data failure rolls back a verified code deploy — is gone: the promote never fails on data, and the refresh still happens, triggered by every successful promote.
• Serve stale with an annotation (the draft's first shape): rejected by the ruling — it hides the problem instead of solving it ("we do not hide our heads in sand"). Staleness is surfaced and the refresh is made to happen; a reader that cannot get fresh data says so, it does not pretend.
• Refresh only on a timer, no read-time check: reintroduces the unbounded staleness window the read-time check exists to close.

Consequences

On Accepted: packet P6c of plans/ast-daemon-deployment-plan.md rebuilds as "promote-triggered semantic refresh with a read-time backstop": the marker write in semantic_graph/ingestion.py, the post-promote trigger step in the ship/runner (not a success gate), the freshness check + trigger in a shared helper the three readers call, the single-flight refresh unit, the re-homed provider probe, and stages_graphs.py switching to --kg. Real-fleet tests prove: a promote with zero embedding calls and no memgraph stage, whose success triggers exactly one refresh the promote does not wait on; a 402 defers, alerts, and is retried by the safety net without failing the reader or the promote; an unchanged tree's refresh embeds nothing (ADR-0074's consequences' tests, moved to the refresh path); a stale reader triggers one refresh and reports the staleness loudly — never a quiet answer from stale data. ADR-0069 applies (@traces, RTM, docs, wiki, ≥90% coverage, one ship). ADR-0075 is unaffected.

Provenance

Mohamed's ruling, 2026-10-07, verbatim: "no separate code promotion … it works in production from Data needs refresh, if data needs refresh, when code runs it checks for freshness and triggers refresh if needed" (scope clarification: "Semantic/RAG only"). Amended after the ruling's clarification, verbatim: "no suppression we promote succesfully and trigger a embedding refresh on succesful promotion, we do not hide our heads in sand we solve the problem correctly" — the first draft's serve-stale-with-annotation shape was rejected; a successful promote triggers the refresh, and staleness is surfaced, never hidden. Forcing facts: the c8c4ddd23 promote's graphs-stage 402 after a passed verify (p6a_ship4.log), the 570-call / 835.6 s p6a_ship3.log measurement, and the 31-project workspace ingest's existing --skip-ingest precedent. Code: promotion/stages_graphs.py:25-82, semantic_graph/ingestion.py:134-199, 271-313, scripts/bootstrap_graphs.py (--kg/--memgraph/--all). Awaiting Mohamed's ruling; only he rules.

Related ADRs

• Supersedes: ADR-0074
• Also mentioned: ADR-0069, ADR-0075

Rulings cited

None.

Specs cited

None (or no spec page exists for the ones named).