Orchestrata Ultimate › Plans › graph-rename-plan

Graph Rename Plan — eradicate neo4j vocabulary, technology- and platform-agnostic

Source: plans/graph-rename-plan.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-27 · 409 lines.

Prose pending.

Plan text

Graph Rename Plan — eradicate neo4j vocabulary, technology- and platform-agnostic

Status: DRAFT — awaiting Mohamed's ratification Date: 2026-07-27 · Session 255 Directive (verbatim): "you must rename neo4j to graph, I want graph, technology and platform agnostic" Execution model: Claude Opus 5 orchestrates (NO edits), work delegated to Kimi (IRC), MiniMax M3, and Claude subagents in this window.

1. Diagnosis — where we are and why

1.1 Current state (measured 2026-07-27 11:47)

1.2 The 4 remaining failures — all one defect class

Every failure is a test pinning a retired string against code already renamed. Zero production failures.

1.3 Why this has taken >1 day (honest accounting)

1. Three entangled waves executed serially in one window: physical decommission (irreversible, careful), env-seam retirement (NEO4J_*→KG_*, ~92 files), and vocabulary (gw→slot).
2. Every verification cycle is a ~6-minute full suite (1455 modules, real infra, kernel gRPC preflight, six gates).
3. Two self-inflicted regressions from half-applied renames (the gw→slot rename broke 4 unrenamed Redis slot parsers → 54 failures → diagnose → fix → re-run). The lesson is encoded below: every rename task owns all references to its symbols, verified by a zero-refs gate, not by eyeball.
4. Live verification (gateway/kernel restarts, production graph queries) is not skippable (RUL-012) and is serial.

The fix for the timeline is parallelism: disjoint task packages, multiple agents, serialized merges, one authoritative gate at the end.

1.4 Remaining neo4j footprint (measured)

• 554 files mention neo4j (src 152, tests 328, scripts 57, rtm 17)
• 171 lines are import neo4j / from neo4j import — the PyPI Bolt driver package, an external name we cannot and should not rename
• 19 modules named after the retired server (files listed per package below)
• 13 class Neo4j* definitions in src
• 676 references in tests to NEO4J_URI/USERNAME/PASSWORD/DATABASE — nearly all via tests/config.py::TestSettings field names
• .env.example still documents the destroyed fleet (NEO4J_URI_GW1..7, dead IPs)
• Docs: INFRASTRUCTURE.md, AGENTS.md, docs/DIAGNOSTIC_PLAYBOOK.md, docs/development-and-testing.md, docs/environment-replication.md, docs/vizint.md, docs/memgraph-ha-cluster.md, docs/diagrams/environment-physical.dot, more

2. Naming policy (the rules every agent applies)

1. Project-owned vocabulary becomes graph. Module names, class names, function names, error strings, report headers, CLI help, env-check messages, doc prose describing current infrastructure.
2. The PyPI package neo4j stays. from neo4j import GraphDatabase is the Bolt driver library — an external name. It remains, confined behind the access layer. Never touch an import of the neo4j package itself.
3. Adapters are named for the protocol, not the vendor. core/graph/adapters/neo4j_{async,sync}.py wrap the Bolt driver → become bolt_{async,sync}.py, classes BoltAsyncRepository/BoltSyncRepository. Bolt is spoken by Neo4j, Memgraph, and orchgraph alike — that IS the technology-agnostic name for a driver adapter.
4. Env keys stay KG_*. KG_ROUTING_URI, KG_URI, KG_DATABASE, KG_ENGINE, KG_WORKER_COUNT contain no vendor name; they were the agnostic seam this whole migration built. No churn.
5. Historical prose stays. Comments/docstrings explaining the decommission ("retired with the Neo4j containers on 2026-07-27") are history, not live-infra claims — keep them. Session-numbered historical docs (docs/session-157-*.md) keep their period wording.
6. KG_ENGINE values stay engine names ("memgraph"). Selecting an engine is the one place technology names are the correct vocabulary.

Grep acceptance gate per package (what "done" means): in every file the package owns, grep -in 'neo4j' matches ONLY: (a) import neo4j/from neo4j driver lines, (b) historical prose referencing the decommission/migration, (c) neo4j:// URI scheme literals (the routing scheme the driver defines — external, like the package name).

3. Execution model

3.1 Roles

• Orchestrator (Claude Opus 5, this window): dispatches, merges serially, runs gates and the authoritative suite, reports. Makes no edits. Conflict resolution and gate-failure fixes are delegated back to the owning agent or a subagent.
• Claude subagents (this window): the two highest-blast-radius packages (P1, P2) + the final sweep (P7).
• Kimi (IRC localhost:6667 #agents, worktree protocol): P3, P6.
• MiniMax M3: P4, P5. Briefs are fully self-contained below; dispatch via Mohamed's harness of choice (opencode/OpenRouter).

3.2 Sequencing

P0 (land pending wave on feat/graph-rename, push)      ← gate for everything
   ├── P1  core access layer          (Claude subagent A)   ─┐
   ├── P2  knowledge_graph package    (Claude subagent B)    │ parallel,
   ├── P3  orchestrator + kernel audit (Kimi #1)             │ disjoint
   ├── P4  session/memory/harness     (MiniMax #1)           │ ownership
   ├── P5  scripts family + 3 failing tests (MiniMax #2)     │
   └── P6  retire migration one-shots (Kimi #2)             ─┘
P7 (tests/config 676 refs + .env.example + docs + LEVEL0)   ← after P1–P6 merge
FINAL GATE (full suite 1455/1455 + rulings gates) → merge to master → push → wiki/session close

Merge queue order: P6 → P1 → P2 → P3 → P4 → P5 → P7. Orchestrator merges into feat/graph-rename one at a time, running each package's targeted tests post-merge. A branch that conflicts is rebased by its owner (or a spawned subagent), never by the orchestrator's hands.

3.3 Worktree protocol (Kimi / MiniMax — MANDATORY preamble in every brief)

cd ~/workspace/orchestrata-ultimate-architecture
git fetch origin
git worktree add -b task/<PKG-ID> ../wt-<PKG-ID> origin/feat/graph-rename
cd ../wt-<PKG-ID>
ln -sfn ../orchestrata-ultimate-architecture/.venv .venv     # share the main venv
export PYTHONPATH=$(pwd)/src                                  # editable .pth points at main repo

• Verify with fast targeted pytest on the modules you touched, not the full suite (the orchestrator runs the authoritative gate). Run per-module pytest processes in parallel in a single command — never sequential waits.
• Commit on YOUR branch only; check git branch --show-current before every commit.
• git commit --no-verify is permitted in worktrees (pre-commit hangs venv-less; the orchestrator's merge gate re-runs everything).
• NO pytest-xdist ever (-n, --dist banned). NO mocks/stubs/monkeypatch/skips. No os.environ["X"] — only .get. No AI attribution in commits.
• The repo auto-formats on write (black/ruff hook) — re-read a file before a follow-up edit to the same region.
• Deliverable per package: pushed branch task/<PKG-ID> + a report: files renamed, refs updated (count), tests run + verdicts, leftover-grep output (must be empty per §2 gate).

3.4 Rename mechanics (all agents)

Use the AST toolkit, not sed, for symbol renames:

python scripts/serialize_codebase_ast.py refs <OldName>                       # before: map the blast radius
python scripts/serialize_codebase_ast.py rename --old <Old> --new <New> --kind class   # dry-run
python scripts/serialize_codebase_ast.py rename --old <Old> --new <New> --kind class --apply

File renames: git mv old.py new.py, then fix every importer (grep old module path repo-wide — your package owns ALL of them, including tests and generated tests). Zero-refs check afterwards: python scripts/serialize_codebase_ast.py refs <OldName> must return nothing, and grep -rn --include='*.py' '<old_module_path>' src scripts tests rtm must return nothing.

4. Task packages

P0 — Land the pending wave (orchestrator + gates; prerequisite for all)

The 107-file KG_* wave exists only in the main working tree. Worktrees branch from origin/* — without P0 every delegated task edits a stale base and the work no-ops. (Learned the hard way; recorded in memory worktree-base-is-origin-master.)

1. git checkout -b feat/graph-rename (master stays clean at 0dc0af6b).
2. Run the deterministic review gates (RUL-008): ruff check src tests scripts, python scripts/audit_test_purity.py, python scripts/test_isolation_audit.py, python scripts/architecture_checker.py, python scripts/serialize_codebase_ast.py standards, python scripts/serialize_codebase_ast.py check-deps. Gate failures → delegated to a lint-fixer/builder subagent, NOT fixed by the orchestrator.
3. Commit the wave (message documents: env-seam retirement, slot rename, 1451/1455 with the 4 vocabulary failures explicitly listed as in-flight → fixed by P2/P5/P6 below). Push.

P1 — Core graph access layer (Claude subagent A — highest blast radius)

Owns: src/orchestrata_ultimate/core/neo4j_factory.py, core/graph/repository.py, core/graph/adapters/neo4j_async.py, core/graph/adapters/neo4j_sync.py, ALL importers of these modules/symbols (82 files import neo4j_factory; 64 files reference Neo4jSyncRepository, 26 Neo4jAsyncRepository), and their test modules.

Function names inside graph_factory (resolve_uri, resolve_kg_database, NO_AUTH, _resolve_auth) are already agnostic — keep. Keep the historical error-string prose explaining NEO4J_URI's retirement (it is accurate history; §2.5).

Investigation item (report, then act): core/graph/repository.py has _BACKENDS = {"neo4j": ...} and _DEFAULT_BACKEND = "neo4j". Determine every caller that passes/receives a backend key (grep create_graph_repository, create_async_graph_repository, backend/engine kwargs, KG_ENGINE consumers: conftest.py, neo4j_factory.py, knowledge_graph/sync.py, start_test_services.py, sweep_test_pollution.py, validate_env.py, kg_schema.py, code-review.py, diagnose_test_failure.py, sync_ast_to_neo4j.py). Rename the backend key "neo4j" → "bolt" only if no external config/env value supplies "neo4j"; if one does, accept both and log the alias for later removal. KG_ENGINE values themselves stay per §2.6.

Trap: do NOT touch from neo4j import ... driver lines; the adapters keep importing the driver. Docstrings referencing "an existing neo4j.Driver" refer to the driver class — keep.

Verify: targeted pytest on all renamed test modules + tests/core/graph/ + a sample of the biggest importers (tests/knowledge_graph/test_kg_sync.py, tests/gateway/test_config.py); zero-refs + grep gates (§2, §3.4).

P2 — knowledge_graph package (Claude subagent B)

Owns: knowledge_graph/neo4j_client.py, neo4j_node_mixins.py, sync.py, config.py, all importers of their symbols, matching tests.

TRAP — duplicate class name: there are TWO Neo4jConfig classes: knowledge_graph/config.py:50 (yours → GraphClientConfig) and orchestrator/neo4j_audit.py:81 (P3's → GraphAuditConfig). A name-match rename would merge them. Resolve refs per-import-site (refs + check the import path), never by bare token. Coordinate: you rename ONLY references importing from knowledge_graph.

Internal method/helper names mentioning neo4j inside these files (e.g. _neo4j_* helpers, _is_transient_error comments) → graph/protocol-neutral equivalents where they name the server; driver-API references stay.

Verify: the whole tests/knowledge_graph/ directory targeted (parallel per-module in one command), zero-refs + grep gates.

P3 — Orchestrator + kernel audit (Kimi)

Owns: orchestrator/neo4j_audit.py, orchestrator/fact_neo4j.py, kernel/audit/writers.py (one class), their importers + tests.

Same duplicate-Neo4jConfig trap as P2, from the other side: rename ONLY references that import from orchestrator.neo4j_audit/orchestrator.graph_audit. Also sweep orchestrator/fact_store.py, fact_manager.py, neo4j_audit importers listed by grep -rln 'neo4j_audit\|fact_neo4j' src tests scripts.

Verify: tests/orchestrator/ audit+fact modules, tests/kernel/test_audit_*, tests/integration/test_graph_audit_integration.py; zero-refs + grep gates.

P4 — Session, memory, harness stores (MiniMax M3 #1)

Owns: session/neo4j_store.py, memory/cold_store.py, harness/history.py, importers + tests.

Also sweep session/kg_sync/, session/kg_context.py, session/sync.py, kernel/session_manager.py, memory/memory_engine.py for neo4j-named locals/strings (the env keys were already migrated; this is identifier/prose vocabulary).

Verify: tests/session/ store+sync modules, tests/memory/test_cold_store*.py, tests/harness/test_history.py; zero-refs + grep gates.

P5 — Scripts family + the 3 failing script-tests (MiniMax M3 #2)

Owns: the server-named scripts, their tests, and three of the four current failures.

Failure fixes (root-caused in §1.2):

1. tests/scripts/test_health_check.py:336–349 — the dropped-key test now drops KG_ROUTING_URI and asserts "KG_ROUTING_URI" in result.stdout.
2. tests/scripts/test_repair_divergent_volk_aliases.py:210–235 — rewrite configured_uris expectations to the KG keys the script now reads (see scripts/repair_divergent_volk_aliases.py:83,217–222); assert the new message "No graph URI configured".
3. scripts/diagnose_test_failure.py:371 — header NEO4J CONSTRAINTS: → GRAPH CONSTRAINTS:; fix tests/scripts/test_diagnose_test_failure.py:348,352 to the new header. Real gap: on per-slot orchkg_w<N> databases the constraint list is empty (production orchkg has 78) — the section must print GRAPH CONSTRAINTS: (none on database <name>) when empty so the report never silently omits the section, and the test asserts the header on whichever database the run targets, not a row count.

Also update every cross-reference to the renamed script paths: grep 'neo4j_query\|neo4j_test_preflight\|sync_ast_to_neo4j\|sync_traces_to_neo4j\|sync_architecture_to_neo4j\|migrate_neo4j_canonical' across src scripts tests rtm docs .claude — includes scripts/gen_level0.py (the "Query:" line), scripts/check_kg_connectivity.py, scripts/kg_dialect_preflight.py (P6 may delete it — coordinate via merge order: P6 merges FIRST, so re-fetch before finishing), AGENTS.md/docs mentions are P7's EXCEPT the literal script-path strings inside scripts/*.py which are yours. Fix the stale --uri bolt://10.92.53.13:7687 docstring examples in backfill_thread_links.py, check_kg_connectivity.py, kg_schema.py, sync_traces_to_graph.py → routing URI neo4j://10.92.53.160:7691 (scheme literal is fine per §2 gate).

Verify: all renamed test modules + test_health_check.py, test_repair_divergent_volk_aliases.py, test_diagnose_test_failure.py, test_backfill_thread_links.py, test_check_kg_connectivity.py; zero-refs + grep gates.

P6 — Retire the migration one-shots (Kimi — merges FIRST)

The migration is complete and irreversible; these tools' inputs (the Neo4j servers) no longer exist. Renaming a tool that can never run again is polish on a corpse — delete (git history retains them).

Delete (script + its test):

• scripts/import_neo4j_to_memgraph.py + tests/scripts/test_import_neo4j_to_memgraph.py
• scripts/migrate_kg_to_orchgraph.py + tests/scripts/test_migrate_kg_to_orchgraph.py
• scripts/kg_query_parity.py + tests/scripts/test_kg_query_parity.py ← dissolves failure #4
• scripts/kg_dialect_preflight.py + tests/scripts/test_kg_dialect_preflight.py
• scripts/gen_kg_migration_inventory.py + tests/scripts/test_gen_kg_migration_inventory.py
• scripts/purge_legacy_orphans.py (zero importers) + its test if present
• plans/kg-migration-inventory.json if regenerated-only

Tangles to resolve (measured, not optional):

• tests/core/graph/test_retry.py imports from kg_dialect_preflight — relocate the helper it uses into the test module or a kept utility; do not delete blind.
• tests/scripts/test_benchmark_code_intel.py imports from kg_query_parity — same treatment.
• import_neo4j_to_memgraph is imported by migrate_kg_to_orchgraph (both deleted — fine).
• Do NOT delete migrate_neo4j_canonical.py — it is live shared code; P5 renames it.
• Use git rm (not rm) and -f if unstaged edits block it (a plain git rm failed silently on exactly this once — check the exit).

Verify: tests/core/graph/test_retry.py + tests/scripts/test_benchmark_code_intel.py pass; grep -rln 'kg_query_parity\|kg_dialect_preflight\|import_neo4j_to_memgraph\|migrate_kg_to_orchgraph\|gen_kg_migration_inventory\|purge_legacy_orphans' src scripts tests rtm → empty.

P7 — Test-config fields, .env.example, docs, LEVEL0 (Claude subagent C — after P1–P6 merge)

Owns: the widest mechanical sweep + the doc surfaces.

1. tests/config.py field rename (676 references across tests): NEO4J_URI→GRAPH_URI, NEO4J_USERNAME→GRAPH_USERNAME, NEO4J_PASSWORD→GRAPH_PASSWORD, NEO4J_DATABASE→GRAPH_DATABASE, TEST_NEO4J_*→TEST_GRAPH_*. Keep the existing validation_alias values (KG_ROUTING_URI, KG_UNUSED_*, KG_DATABASE) exactly — they are the env seam and are already agnostic. Update every settings.NEO4J_* / TestSettings().NEO4J_* reference; word-boundary precise (\bNEO4J_URI\b), then ruff check tests + grep -rn 'NEO4J_' tests → only historical comments remain.
2. tests/infrastructure/__init__.py: NEO4J_USERNAME/NEO4J_PASSWORD consts → GRAPH_* (+ importers).
3. .env.example: delete lines 22–24, 34–37 block, 60–62, 65–74 (NEO4J_*, GW fleet, dead IPs 10.92.53.11/.13/.14/.19); document the real seam: KG_ROUTING_URI, KG_URI, KG_DATABASE, KG_ENGINE, KG_WORKER_COUNT, TEST_MEMGRAPH_URI + test node 10.92.53.167. Mirror the live .env key-for-key (values placeholdered).
4. Docs (current-state only; historical session docs keep period wording): INFRASTRUCTURE.md, AGENTS.md, docs/DIAGNOSTIC_PLAYBOOK.md, docs/development-and-testing.md, docs/environment-replication.md, docs/vizint.md, docs/memgraph-ha-cluster.md, docs/session-and-kg-sync.md, docs/SESSION_CONTEXT.md, docs/diagrams/environment-physical.dot — remove destroyed-fleet references, rename script paths to their P5 names, NEO4J_* keys → KG_*.
5. scripts/gen_level0.py final vocabulary pass, regenerate docs/LEVEL0.md, and python scripts/gen_level0.py --check → 0.
6. Leftover sweep: grep -rin 'neo4j' src scripts tests rtm docs .claude --include='*.py' --include='*.md' --include='*.dot' — classify every remaining hit against the §2 gate (driver import / historical / neo4j:// scheme); fix everything else. Include src/orchestrata_ultimate/prolog/test_schedule.pl.

Verify: full tests/scripts/test_gen_level0.py, tests/core/test_claude_hooks_config.py, a targeted run of the 20 most-affected test modules, gen_level0 --check, grep gate.

5. Final gate & close-out (orchestrator)

1. All packages merged into feat/graph-rename.
2. Authoritative full suite — python scripts/run_tests.py with kernel required (never --skip kernel), JSON via redirect (> reports/run-tests-graph-rename.json — --json is store_true), exit code captured bare (never through a pipe). Required: 1455/1455.
3. Rulings gates: ruff, purity/isolation audits, architecture_checker.py, standards, check-deps, rulings_lint.py, pyright_ratchet.py, gen_level0 --check.
4. Repo-wide acceptance grep (§2) — every hit classified allowed.
5. Live verification (RUL-012): gateway + kernel restart clean under systemd; one production query through scripts/graph_query.py against orchkg.
6. Reviewer subagent pass over the merged diff (RUL-008), fixes delegated.
7. Merge feat/graph-rename → master, push (glab credential-helper path, not the stale GITLAB_TOKEN).
8. Wiki page update + wiki-kimi validate + index + wiki-rag-ingest; memory-file updates (orchestrata-graph-infra, AST/tooling references); session CLI work-log, handoff, end.

6. Decision log (Mohamed may veto any of these before dispatch)

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks