Home / Projects / Orchestrata Ultimate / Decisions / ADR index / ADR-0109
ADR-0109: Semantic-graph bulk writes run in transactions of at most 500 rows; replication stays SYNC
| field | value |
|---|---|
| Status | Accepted — Accepted (ruling, Mohamed, 2026-10-19, verbatim: "Batch the big writes") |
| Date | 2026-10-19 |
| Register | factory (docs/adr, 4-digit ids) |
| Source | docs/adr/0109-semantic-graph-bulk-writes-run-in-transactions-of-at-most-500-rows-replication-stays-sync.md — permalink at c2623c2e4 (unchanged on master since that commit) |
| Wiki register entry | ADR-0109 (wiki status: Accepted) |
The record
Status: Accepted (ruling, Mohamed, 2026-10-19, verbatim: "Batch the big writes") · Date: 2026-10-19
Context
The prod promote of d4037442 failed at the graphs stage: bootstrap_graphs.py --all →
ingest_orchestrata_ultimate exited 1. Prod orchgraph's main (instance_1) logged "Couldn't
replicate data to instance_2 because timeout occurred": MemgraphSemanticGraph.clear() ran
MATCH (c:CodestralChunk) DETACH DELETE c as ONE transaction over every chunk (each with a
1536-dim embedding). The SYNC replica took ~54 s to apply it; the replication RPC waits a fixed
5 s. link_chunks_to_modules() had the same unbounded MATCH-all shape. The delete committed on
main, leaving prod's RAG corpus empty.
Decision (Mohamed, verbatim: "Batch the big writes")
Semantic-graph bulk writes are bounded to transactions of at most 500 rows
(WRITE_BATCH_SIZE, overridable per adapter instance): clear(), the chunk-label delete in
clear_prefixed(), and link_chunks_to_modules() each loop a WITH … LIMIT $batch write, one
transaction per pass, until a pass touches 0 rows. Replication stays SYNC.
Alternatives rejected
- ASYNC replication / a longer replication timeout — trades durability or hides the cause; any future unbounded write would hit the same wall at a larger corpus.
- Drop and recreate the database instead of deleting — the semantic DB is shared with the knowledge-wiki project; a whole-DB reset destroys its data.
Consequences
A replica applies at most 500 chunk deletes/links per transaction. A re-run of
link_chunks_to_modules() over a linked corpus is one empty pass (it only matches chunks with no
BELONGS_TO). Enforced by tests/semantic_graph/test_memgraph_adapter.py (batch=2 tests that
can only pass by looping).
Provenance
Mohamed's in-session ruling 2026-10-19; root cause from the prod main's journal at 09:14:47Z
(promote of d4037442). Implementation: src/orchestrata_ultimate/semantic_graph/adapters/memgraph.py.
Related ADRs
No other ADR is named in this record.
Rulings cited
None.
Specs cited
None (or no spec page exists for the ones named).