Knowledge Wiki Orchestrata Ultimate

Orchestrata Ultimate › Plans › graph-consolidation-decision

Graph-consolidation decision — operational KG on orchgraph vs two-graph topology

Source: plans/graph-consolidation-decision.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-12 · 113 lines.

Prose pending.

Plan text

Graph-consolidation decision — operational KG on orchgraph vs two-graph topology

Session 227 (2026-07-12). Evidence: read-only Cypher-surface audit of the operational-KG code + live construct probes against the orchgraph cluster (neo4j://10.92.53.160:7691)

Recommendation

Keep the two-graph topology today. Consolidation is FEASIBLE (medium-large mechanical effort) but is blocked by five ordered prerequisites, four of which are already backlog items. Re-decide after the prerequisites land; do not start a cutover before them — migrating the current schema would carry known split-brain debt onto a new engine and compound it (ontology-before-import rule).

Evidence

What works on orchgraph today (live-probed 2026-07-12)

Construct Verdict
datetime(), localDateTime() OK (tz-aware DateTime returned)
EXISTS {} subqueries, FOREACH, SET n += map OK
elementId(), id() OK
Label expressions MATCH (n:A|B) OK
CREATE CONSTRAINT ... IF NOT EXISTS ... REQUIRE ... IS UNIQUE OK (proven by semantic_graph adapter)
Plain MERGE, DETACH DELETE, variable-length multi-type patterns OK
SHOW INDEX INFO / SHOW CONSTRAINT INFO OK but different row shape than Neo4j SHOW INDEXES/CONSTRAINTS

Zero occurrences in the operational write path of: CALL {} subqueries, CALL ... IN TRANSACTIONS, ON CREATE/ON MATCH SET, write-path apoc.*.

Confirmed incompatibilities (each needs code changes before cutover)

  1. Schema DDL: CREATE INDEX name IF NOT EXISTS FOR (n:L) ON (n.p) and CREATE FULLTEXT INDEX (knowledge_graph/schema.py:147-174) are Neo4j-5-only; Memgraph wants CREATE INDEX ON :L(p) (no IF NOT EXISTS; duplicates raise).
  2. Fulltext search: CALL db.index.fulltext.queryNodes (knowledge_graph/search.py:48, session/kg_context.py:229) has no counterpart — the current orchgraph build ships vector_search.* but no text-search MAGE module (probed: none loaded).
  3. apoc.path.subgraphAll (knowledge_graph/kg_context.py:246) — no APOC on Memgraph; needs a MAGE or variable-length-MATCH rewrite with parity tests.
  4. Introspection consumers: canonical_schema.py:516-524 _live_unique_pairs(), scripts/migrate_neo4j_canonical.py:224, scripts/neo4j_test_preflight.py:88 parse the Neo4j-5 SHOW CONSTRAINTS row shape (entityType/labelsOrTypes/...) — on Memgraph they silently return nothing rather than erroring.
  5. CALL db.labels() / db.relationshipTypes() (knowledge_graph/_sync_queries.py:200,209) — procedures absent (probed: FAIL).

Silent-failure hazard (fix regardless of the decision)

knowledge_graph/neo4j_client.py:117-129 ensure_schema() wraps all DDL in try/except Exception: logger.debug(...). On Memgraph the incompatible index/fulltext DDL would be swallowed and the KG would boot with no indexes and no fulltext, silently. Also: ensure_schema() applies the legacy schema.py DDL while scripts/kg_schema.py / migrate_neo4j_canonical.py apply canonical_schema.get_all_ddl() — two DDL sources of truth (kg-schema-gaps gap #4).

Seam status

Tests

tests/conftest.py already indirects NEO4J_URI (+ per-worker NEO4J_URI_<SLOT>), so the switch point is one env var — but ~237 non-semantic-graph test files touch Neo4j and every Neo4j-behavior-dependent assertion (DDL sets, SHOW shapes, fulltext, id semantics) needs re-validation. test_schema_completeness.py / test_canonical_schema_completeness.py assert exact DDL statement sets and must change in lockstep.

Capacity / operations

Ordered prerequisites (if consolidation is pursued)

  1. Fix ensure_schema() fail-silent + unify DDL onto canonical_schema.py (also closes kg-schema-gaps gap #4). Independent win; do it regardless.
  2. Land the persistence funnel work already in backlog: D12 P1 ratchet + the Neo4j repo-seam swarm-refactor, and bring the 9 bypassing scripts through the factories.
  3. Fix kg-schema-gaps #1 (Task node), #2 (unwired provenance writers), #8 (Volk MERGE-key divergence) — never import a known-split-brain ontology onto a new engine.
  4. Solve fulltext: build/load a text-search MAGE module into the orchgraph fork, or redesign search.py + session/kg_context.py retrieval on vector_search.
  5. Define telemetry retention (age-out/archival for GatewayAuditEvent/CXPPacket/Session) before moving append-only growth into RAM.

Then: dual-write or import-through-ontology cutover behind the existing NEO4J_URI indirection, per-worker test validation, and a rollback window keeping .11 warm.

Prior decisions honored