Knowledge Wiki Orchestrata Ultimate

Orchestrata Ultimate › Plans › parallel-collision-fix-plan

Parallel-collision fix plan — suite failures from shared port + File/Directory constraint races

Source: plans/parallel-collision-fix-plan.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-22 · 149 lines.

Prose pending.

Plan text

Parallel-collision fix plan — suite failures from shared port + File/Directory constraint races

Origin: 2026-07-22 full-suite measurement (session 252): 575.5 s, 27,187 passed / 41 failed / 15 errors across 19 modules. Both failed-module samples pass solo (9 passed, 6 s) — the failures are concurrency/order collisions, not broken code. Evidence: scratchpad full_suite_run2.json (session 12166b56), failure text in session-252 work log.

Two verified failure classes:

  1. KG gRPC port race — KG_GRPC_PORT=50052 is one fixed value shared by all 4 batch subprocesses; any two modules that bind it concurrently collide (Failed to bind to address 0.0.0.0:50052 in tests/knowledge_graph/test_server.py). Precedent for the fix already exists: tests/conftest.py:56-72 remaps NEO4J_URI per ORCHESTRATA_WORKER_SLOT (NEO4J_URI_GW<N>); the port needs the same treatment.
  2. File/Directory MERGE-key ≠ constraint-key — sync code MERGEs on {id: ...} (session/kg_sync/_stage_folders.py:112,125, knowledge_graph/synchronizer.py:321, knowledge_graph/kg_context.py:718) while the Neo4j uniqueness constraint is on path (scripts/test_isolation_audit.py:153 documents this). Two nodes with different ids and the same path → ConstraintValidationFailed (File path='context/assembler.py', Directory path='.'). _stage_code.py:88 already documents that path is "unsound as a global MERGE key" — the contradiction is live in production code, not only in tests.

Workstream A — per-slot KG_GRPC_PORT (mechanical wiring; no design fork)

A1. Port derivation in conftest — in tests/conftest.py, next to the existing NEO4J_URI_GW<N> remap (lines 56-72), derive KG_GRPC_PORT = base + <slot-index> from ORCHESTRATA_WORKER_SLOT (gw0 → base, gw1 → base+1, …), set once at conftest import (same "session constant, not per-test mutation" discipline as the Neo4j remap; regression precedent: tests/core/test_worker_neo4j_mapping.py). No bare os.environ[...] reads; follow the .get() rule. A2. Regression test — mirror test_worker_neo4j_mapping.py: two subprocesses with slots gw0/gw1 resolve distinct ports; a batch with no slot falls back to the base value. A3. Consumer sweep — src/orchestrata_ultimate/knowledge_graph/server.py, scripts/validate_env.py, tests/scripts/test_validate_env.py, docs that state 50052: confirm all read the env var (not the literal) and none assert the fixed value. Update core/threading/test_executor.py docstring (currently documents only the Redis-slot purpose).

Assignee: Sonnet wirer agent (surgical connect-A-to-B change + its tests; stops on any pre-existing debt it bumps into — exactly its charter). Independent of Workstream B; can start immediately.

Workstream B — File/Directory identity (design ruling FIRST, then build)

B0. Canonical-schema ruling — Opus (orchestrator), NOT delegatable. Resolve the MERGE-key vs constraint-key contradiction against SPEC-106 canonical_schema.py + plans/kg-schema-gaps.md (specs are authoritative over tests — standing project rule). The three candidate rulings:

Workstream C — census + verification sweeps (Kimi)

Mechanical, high-volume, judgment-light classification — Kimi-tier per the model-tiering rule (analogous to the Haiku charter: extraction/classification, not bug-fixing).

C1. Failure census — for each of the 19 failed modules in full_suite_run2.json (module stdout/stderr captured inline): classify into {port-race, File/Directory-constraint, other}. Deliverable: a table appended to this plan. Acceptance: every one of the 41 failures + 15 errors classified; any "other" escalates to Opus before B1 lands (a third failure class must block the "collision-only" conclusion). C2. Repo-wide twin-bug sweep — two enumerations: (i) every fixed port shared across batch subprocesses (gateway 8765, kernel 50053 are shared infra BY DESIGN — flag only ports that tests themselves bind); (ii) every label in canonical_schema.py carrying a uniqueness constraint, cross-referenced against every MERGE (:<Label> {…}) site whose merge key differs from the constraint key — the File/Directory bug's siblings. Deliverable: list with file:line, input to B0. C3. Post-fix stability verification — after A+B merge: 3 consecutive run_tests.py --json runs (launched by the orchestrator; Kimi diffs the three JSONs): 0 occurrences of either failure class, wall time within 575 s ± 15%. Any residual failure → classified and routed back.

Kimi dispatch: cd ~/workspace/orchestrata-ultimate-architecture && kimi --auto with the per-task brief (C1/C2 briefs to be written as plans/briefs/kimi-c1.md, kimi-c2.md); project convention scripts/kimi-startup.sh first. IRC #agents (scripts/_irc_send.py) for completion signaling per SPEC-021 producer convention.

Sequencing & gates

A1→A2→A3 (wirer)  ──────────────┐
C1, C2 (kimi, parallel) ──→ B0 (Opus ruling) ──→ B1+B3 (builder) ──→ B2 (test-writer) ──┤
                                                                                        └──→ C3 (3× suite) → commit

C1 census outcome (2026-07-22 04:15) — third failure class confirmed and triaged

Kimi C1 flagged ESCALATE: third failure class present — correctly. The "collision-only" conclusion was wrong. Census (plans/briefs/kimi-c1-result.md): 2 port-race, 6 file-dir-constraint, 27 other (kimi's row total does not reconcile to the 41+15 run totals — rows appear grouped; the qualitative classes were verified by solo reruns).

Triage of "other" by deterministic-vs-load split (solo rerun evidence):

C3 round 1 outcome (07:26–07:45) — collision classes dead; my port formula had a defect, fixed

Runs 1–2 on the migrated DBs: 27,245/23/0 in 459.3 s, then 27,239/24/5 in 447.4 s — zero Failed to bind and zero ConstraintValidationFailed across both. Remaining failures are the load-class watch-list (gateway nested-pytest runners, governance handler, script-runner tests) — next workstream, not collision class.

Run 2 exposed a defect in the Workstream A spec (mine, not the wirer's): base+N put slot gw1 on 50053 — the kernel's CXP_GRPC_PORT — manifesting as "port 50053 still held by a previous KG server" setup failures in test_server.py. Fixed: gwN → base+100+N (gw0 keeps base), plus a guard test asserting no derived slot port can equal the kernel port. Note for verification tooling: this collision class signature is "still held by a previous KG server", NOT a bind failure — the C3 diff must grep both. C3 restarted clean (07:45).

Explicit non-goals