Orchestrata Ultimate › Sessions › Session 242

Session 242

‹ previous · next › · range page · timeline

Prose pending.

Purpose

Post-push follow-up on kg-schema-remediation: MR prep / backlog decisions / new direction

Work log

1. Retrieved latest: MR merged (26579c4d), master fast-forwarded to 3b159679 (Prolog codebase intelligence layer, author Hannah/openclaw.ai); deleted merged local kg-schema-remediation
2. Cloned MSamy/system-design-plugin (the new architect/system-design Claude Code skill, updated today) to ~/workspace/system-design-plugin via glab (env -u GITLAB_TOKEN workaround for expired env token)
3. Wave-1 review complete: D1-D10 all CONFIRMED with live repros + new F1 (loader written against fictional schema — complexity all-1, lines wrong, raises/catches permanently empty, call_line always 0), F6 (stale RESULT_FILE on timeout), F12 (unregistered+untested). Verdict: layer not trustworthy as shipped.
4. Wave-2 complete (worktree agent-a1795c83a6dc572de): F1-F10 fixed — serializer additively emits end_line+complexity (post-processing enrichment in serialize CLI), loader reads real format, dead stmt layer + raises/catches + reaches/2 deleted, JSON regex hack removed (json.loads only), collision-proof goal building, per-instance workdir + attach(), honest PROLOG_STATUS. Gates: ruff 0 (wrapper), pyright 0/0, AST core tests 24/24 x2. Diff held in worktree until benchmark agent finishes (avoids racing the .pl under test).
5. Benchmark complete (parity PASS all 4 workloads, identical result sets across P-sub/P-janus/Py-mem). VERDICT: 'Prolog faster than Python for recursive work' FAILS vs warm in-memory Python recursion (Py-mem wins W1-W4; closest W3 cycles 11.06ms vs janus 19.12ms = 1.7x; subprocess mode loses by 3-5 orders due to ~1.2s consult/call). PASSES vs the cold Python CLI end-to-end: P-sub 3.00x/1.76x faster than call-graph/find-cycles; P-janus 2297x/111x. Extra finding: library(http/json) fails under janus (PL_release_stream ABI mismatch) — future janus json_read_dict use will fail. Report: reports/prolog-codeintel-benchmark.md
6. Fixed orchestrata_ast/analyzer.py plain-import asname loss (single-name imports now carry alias; multi-name unrepresentable stays None by model shape) + 3 regression tests in TestImportAliasCapture; 27/27 green solo, black/ruff/pyright clean
7. Wave-4 merge fallout fixed: SWI dual JSON convention clash — wave-3's @(true) fix satisfied classic json_write but broke dict-mode atom_json_dict consumers (benchmark re-run failed rc=2 'json_term expected, found @(false)'). Canonical resolution: KB reverted to atom booleans + wrapper switched to json_write_dict (atoms->real JSON booleans). Also fixed code-review findings: any-return typed, dep_path long line split, reindex/_run share _read_result helper, _prolog_val static. Benchmark wrapper row updated to post-fix API (attach/workdir).
8. orchgraph installer landed + committed (83b722c in ~/workspace/orchgraph): install/package scripts + INSTALL.md; verified on real host (check-deps PASS pacman, dry-run transcript, 243M tarball sha256 OK, corrupted-checksum fails closed); agent found+fixed EXIT-trap exit-code bug; cleared stale Jul-10 index.lock
9. Janus transport landed+committed: 791x measured (1.491ms vs 1179ms median), fail-fast no-fallback per user ruling (raises on missing janus or GIL-on), parity deep-equal, thread smoke 4x10 clean, 16/16 tests, all gates green
10. Wiki taxonomy cleanup landed+committed in knowledge-wiki: validate CLEAN (was ~50 errors), +20 controlled tags synced cli+taxonomy, 14 pages normalized, 3 YAML unquoted-colon frontmatter bugs + duplicate-frontmatter fixed, indexer _is_ignored_path fix, 121 pages reindexed; indexer parsing tests verified green
11. CORRECTION to previous entry: indexer parsing tests had NOT run (no pytest in either wiki venv — both attempts failed with 'No module named pytest'); re-running with uv --with pytest; the wiki cleanup+commit itself stands, verified by wiki-kimi validate clean + syntax check, but the test-verification claim was premature
12. Indexer parsing tests NOW verified for real: 8/8 passed (uv --with pytest, observer plugin) — the graph-indexer _is_ignored_path fix is confirmed green
13. SKELETON_FLOW structural escape: task.py

• STRUCTURAL: 2 escape(s) A.init: method deleted A.m: method deleted

14. Suite-speed mechanical lane committed: gates parallelized (21s->11-12s measured), LEVEL0 drift regenerated, architecture checker verified clean; 115 tests green across affected modules
15. Lever-1 co-scheduling committed: 6 Memgraph files atomic in LPT; uncontended 6-file total 262s < old 287s single-file contended cost; zero retries/failures; executor 73/73, 97% cov
16. AST daemon landed; wire-daemon criterion met

• Verified 28/28 daemon tests + warm-cache 17/17; root-caused 4 warm-cache failures to a live session daemon intercepting delegating subcommands (local sidecar stats hits=0) and pinned --no-daemon in the module _run helper; docstring/review pass on ast_daemon.py (15->4, remainder ungated mypy FastAPI decorators); committed 54e08092. Replicas .13/.14/.19 all conform to canonical schema post-migration; drift-affected module 28/28.

17. Replica canonical conformance verified

• kg_schema.py check OK on bolt://10.92.53.13/.14/.19 after executed migration; test_context_injector_coverage.py 28/28 green.

18. SKELETON_FLOW structural escape: task.py

• STRUCTURAL: 2 escape(s) A.init: method deleted A.m: method deleted

19. janus installer gained --from-gitlab per user command

• Answered honestly: installer previously had only --wheel/--from-source (no GitLab). Uploaded wheel+sdist+.sha256 to GitLab generic package registry (janus-swi-ft 1.5.3-orchestrata.1, project 81437762) via glab api; added --from-gitlab mode (download + sha256 verify + reuse --wheel path, temp-dir cleanup, env overrides); dry-run PASS; download round-trip sha256 byte-identical (2de49a71...); docs/free-threaded-prolog.md updated; committed. Full end-to-end --from-gitlab install+verify deferred until the replication suite run finishes (keep the measurement box clean).

20. SKELETON_FLOW structural escape: task.py

• STRUCTURAL: 2 escape(s) A.init: method deleted A.m: method deleted

21. Three commanded research reports delivered

• org-mode: REJECT with evidence (reports/research-org-mode.md — no production Python parser, breaks Wiki.js/GitLab mermaid rendering, duplicates session/goal/Todoist subsystems). orchestrata-as-plugin: build MCP server first then Claude Code plugin, defer SPEC-015 self-hosting (reports/research-orchestrata-as-plugin.md). architect skill vs volk: skill is SKILL.md+26 refs only; recommend ontology-ratify->KG ingestion (Option C) then additive prompt fold (A-lite); B rejected (reports/architect-skill-vs-volk-integration.md). Ontology draft delivered to user with 8 open rulings.

22. Five-round wiki review executed (user command)

• R1: 4 broken links fixed, 12+ orphans de-orphaned, performance tag added to controlled vocab. R2: timing/counts/status refreshed across 6 pages (13min->10:12, 24->26 packages, 1.1.1->1.3.0, replication n=2 numbers). R3: 118/118 mermaid blocks PASS under real 8.8.2 harness. R4: TOC/module-index nav completed; NEW modules/context + modules/planning pages authored from source. R5: narrated suite-speedup-story page (4 diagrams) + crash-mechanism/runner-pipeline/Raft-topology diagrams added. Free-threaded prolog page GRADUATED to PRODUCTION slug+framing. Wiki committed d3aeb01; all pages live-verified via wiki get.

23. CodebaseTools registered in unified tool registry (production wiring commit 00f56259)

• 4 tools (impact/module_deps/find_cycles/dep_path) via register_native_tools; lazy fail-fast-preserving handle (RuntimeError verbatim, retried, never cached); ValueError-on-limit defect fixed in same pass; 12/12 new + 182/182 regression x2 + independent 29/29; all gates verbatim green.

24. Architecture ontology documented in the wiki (user command)

• New page projects/orchestrata-ultimate/architecture-ontology (PROPOSED status): narrative why/where-from, 6-phase provenance diagram + full core ontology diagram (both PASS Mermaid 8.8.2 harness), node-family table (21 new/4 ext/6 reuse), design principles, sync-writer supersession, the 8 open rulings condensed. Linked from TOC/index/decisions (decisions gains pending-ratification banner). All pages live-verified via wiki get; validation passed; wiki committed; reindex running.

25. Three KG bugs fixed+committed (cd7036b0) + live-graph repair executed

• ast-to-kg now matches the writer's project-qualified ids (:Function/:Method) — was a production no-op; analyst/compliance resolve canonical slug via project_resolver + KGDesignStore fails fast on empty slug; sweeper file_id fingerprint added. 181/181 across 8 affected test files; ruff/black clean. Live repair via new scripts/repair_project_hubs.py (dry-run gated, allow-listed rels): 11,205 mis-anchored DEFINES/HAS_SPEC edges MERGE-moved onto the canonical orchestrata-ultimate hub, both anonymous Project hubs deleted, VERIFY 11240 outgoing/0 bogus.

26. Branch pushed, MR !3 opened and MERGED to master

• 18 commits merged (759e1f36..d0c3a21f); local master fast-forwarded; verified state:merged via glab mr view.

27. Ontology phase 1 waves 1a/1b/1a-writer landed + DEPLOYED

• 1a schema: 19 labels + Change key, 42 REL_*, 5 indexes, 126/126 x2, committed. 1b Component unification: EXECUTED on all 4 DBs (5 backfilled primary, constraints re-keyed), all conform, 166+107 green, committed. 1a-writer/1c/1d: ArchitectureWriter (100% cov, 35/35) + ADR body_sha guard/gate wired into kg_schema check + rewritten sync CLI; cross-wave reconciliation (2 superseded tests corrected); 155/155 independent; committed. DEPLOYED live: self-model synced in ontology terms (1 Architecture/5 Components/6 Constraints/9 Patterns — Rule-label writes gone), reference seed 5 WAFPillars + 30 CloudServices idempotent; kg_schema check OK after every step. 1e hook wave in flight.

28. Ontology phase 1 MERGED: MR !6 (merge 340f3429) — waves 1a/1b/1a-writer+1c+1d/1e all landed on master

• Phase-wide verification 236/236 observed 0 skipped; kg_schema check OK live; wave 1e = SessionStart full-architecture-sync hook (amended ruling 8): --seed flag, bolt-probe short-circuit, pidfile guard, live-run seed{5,30}+sync{1,5,6,9} exit=0; settings.json SessionStart registration (harness security-warning disclosed, explicitly commissioned)

29. Ontology phase 2 MERGED: MR !7 (merge 87a0e324) — ADR corpus graduation + Memgraph semantic twin, both deployed live

• 20/20 ADRs as ArchitectureDecision (Accepted, guard active, both corpus formats parsed); twin: 20 ADR + 11 design-doc ArchitectureChunk nodes, retrieval proof rank-1 hits; hook seed+adrs+twin end-to-end 0.013s; 198/198 phase tests observed + llm_integration proof; kg_schema OK

30. Debt batch pushed: _FakeClient double removed (real-infra behavioral proof), twin CLI HA-failover retry, 4 pyright errors cleared in test_memgraph_adapter, writer lazy-ownership paths covered

• Commits 203f23b6/be5c1829/d4d5057d/3d45f307; 3+14+50+34 tests observed green; SPEC-008 work-context wire-vs-amend surfaced for user ruling

31. Test-slowness investigation: label-less cleanup sweeps full-scan the DB per test; writer file fixed 24.1s->14.8s; conftest-level sweep surfaced with options

• 803k nodes primary / 230-733k workers; test_id sweep 275-637ms per test across ~500 kg tests; pooling verified correct (session driver, pool 16); fix = CALL(){} UNION over writer's 25 labels, ~6ms cached

32. Committed test-performance + isolation-debt batch in 7 scoped commits (42feb773..987c94a3)

• census label-bounded sweeps
• conftest Option 1 deferred cleanup
• auditor waivers 76->0
• Memgraph adapter session-label refactor -61%
• 10 SUT env seams
• pyright casts + agent stash bans
• research report + provisioning template

33. Fixed Option-1 pollution failure from kg directory run: context_pack seed leaf func_a now test-unique; context_pack 55 passed
34. Corrected session/context.py lock-parity docstring: real counterpart is ~/.claude/tools/orchestrata/context.py; tools/session/context.py is dead legacy
35. Deleted test_search.py (MinimalClient capture double evading purity audit); absorbed unique coverage into test_cov_knowledge_search.py incl. disconnected-real-client path
36. Wired knowledge.* work-context RPCs (f7c851c2) + TEXT-indexed scoped substring search (ffbb1a91); deployed TEXT indexes ONLINE on all 4 Neo4j DBs

• coverage_history/rtm_progress were dead code, now registered
• SearchEngine._contains_search + QueryEngine.search label-less scans fixed
• test_search.py MinimalClient double deleted

37. Neo4j memory sizing deployed: primary heap1g/pagecache2g, workers 1g/1g (was 512m/256m vs 652M store); rolling restart verified all 4 DBs; baked into provision_node.sh
38. User ruling recorded: never skip infra verification (PYTEST_SKIP_SERVICE_START reverted); memory file feedback-never-skip-infra-verification
39. GDS Community 2.13.2 installed on neo4j-db + evaluated: PageRank on live CALLS graph (proj 1.43s, PR 0.33s); provisioning template updated; found CodeFunction carries no CALLS edges (possible sync gap, backlogged)
40. RAG/history sweep: Neo4j code-intel incremental sync (15 changed modules), sessions 238-242 wiki history page authored+published+linked, wiki repo committed a7b26d3; writer chain queued: graph index -> wiki-rag-ingest -> final graph index; semantic-graph source re-ingest to follow
41. Wiki RAG converged onto orchestrata stack (wiki commits 4f21c35, 10dd390): OpenRouterProvider codestral-embed-2505 + cohere/rerank-4-pro reranker in wiki-rag-query (default on), store meta.json with dim-mismatch hard-fail; 8/8 tests incl live embed+rerank; 2929 chunks re-embedded; reranked retrieval verified 0.9475 on deployment table
42. RAG stack evaluation delivered (reports/rag-stack-evaluation.md): 6-config goldset A/B, context-pack benchmark p50 176ms, ruling-2 entity-linker production defect found+fixed (8b072d7d), 7-item enhancement proposal

Artifacts

• plans/prolog-codeintel-verification-plan.md - Session-242 plan: defects D1-D10, 4-way benchmark design W1-W4, swarm waves, risk review
• plans/architecture-ontology-draft.md - DRAFT architecture ontology from system-design plugin: 31 node types (21 new), ~44 relationships, SPEC-106-conformant DDL, Mermaid diagram, 22-MERGE worked example, 8 open questions

Decisions

• Benchmark drives Prolog directly (own subprocess+janus drivers), not through the buggy shipped wrapper; Py-mem baseline recurses over the IDENTICAL edge set dumped from Prolog so resolver differences cannot pollute the recursion comparison: Decouples wave 1 from wave 2 fixes; makes the speed claim testable apples-to-apples; a fast wrong answer must lose, so parity is asserted before timing
• F1 remediation: extend serialize_codebase_ast.py ADDITIVELY (end_line via ast.end_lineno + per-function complexity reusing the toolkit's existing calculator), adapt the Prolog loader to the REAL AST format, and DELETE the dead statement layer (tool_raises/tool_catches/stmt walking) — not fake it: Serializer extension is small and optional-field additive (existing consumers unaffected); raise/except capture would be a large serializer change with no current consumer — honest removal + backlog beats silently-empty tools
• Janus wiring scope: in-process janus becomes the QUERY transport inside CodebaseTools with subprocess fallback; reindex remains subprocess (library(http/json) fails under janus per benchmark); volk tool-registry registration remains the open P2 decision: Captures the 750x measured transport win now without forcing the which-volks/reindex-trigger design decisions; http/json ABI failure makes in-process reindex impossible today
• Ontology RATIFIED — all 8 open questions ruled (MR !4, f0001e3f): 1) ADR immutability = writer guard + body_sha conformance gate (prevent AND detect); 2) Component unified + project-qualified {project}:{name} (no design/runtime split-brain; File/Directory precedent); 3) architecture rules fold into Constraint{kind:invariant}, Rule reserved for Prolog; 4) semantic twin YES as phase 2 (ADR/DesignDoc prose -> Memgraph SemanticChunk keyed by Neo4j id); 5) ArchitectureDecision distinct from Decision (different lifecycles/immutability); 6) CloudService lazy + curated ~30 seed with neutral_kind, full catalog rejected (FORCED_BY doctrine); 7) OpenSpec owns LABEL_CHANGE with id={project}:{change_id}; 8) converged writer BOTH phased a->b — rewrite sync_architecture_to_neo4j.py first, then pipeline/session writers; on-demand runs write immediately, session-start seeds self-model only.
• Ontology ruling 8 AMENDED (user): session start initiates a FULL architecture sync: User ruling 2026-07-19 superseding my seed-only trigger; my cost objection didn't hold at this layer's scale (design corpus is dozens-to-hundreds of nodes vs the code graph, which stays incremental per SPEC-023). Engineering caveats preserved: sync runs BACKGROUNDED so session start never blocks on Neo4j, and ruling 1's ADR-immutability guard is enforced inside the full sync (MERGE everything, never mutate an Accepted ADR body). Recorded: draft (MR !5, f06ac434), wiki ontology + decisions pages (live-verified), backlog phase-1 item covers the hook wiring.
• Twin chunk label = ArchitectureChunk, not the draft parenthetical SemanticChunk: SemanticChunk is in STALE_CHUNK_LABELS and find_stale_chunk_nodes' predicate (id present, no chunk_id, no DERIVED_FROM) would match twin chunks — --prune-stale-labels --apply would delete live ADR chunks. Ruling 4 mandates only the Neo4j-id key; distinct content type gets a distinct label + own vector index.
• component_work_context/volk_work_context: keep as spec-mandated surface; wire-vs-amend needs a user ruling: AST refs: 0 production callers for BOTH; but both @traces to SPEC-008 (REQ-008-005 US-001, REQ-008-012 US-003) so deletion breaks RTM. Their consumer was superseded by the SPEC-106 selector-backed ContextInjector. Options for user: (a) wire them in as a SPEC-106 context-pack provider, (b) amend SPEC-008 to mark the reqs superseded-by-106, (c) accept as tested standby surface.
• Wiki RAG embeddings/reranker converged on the orchestrata semantic-graph stack via OpenRouter: One shared embedding space + one reranker across both knowledge surfaces; stdlib-HTTP sync port of the proven OpenRouterEmbedder/Reranker (no new wiki deps); keyless fallback to local MiniLM preserved; meta.json guard prevents silent cross-model search

Achievements

• Prolog codeintel verification complete: 13 defects found+fixed (adversarial review with live repros), 8 real-swipl tests at 93% coverage, parity-gated 4-way benchmark. VERDICT: Prolog FAILS vs warm in-memory Python recursion (all workloads), PASSES vs cold CLI end-to-end (3-2297x) — the win is persisted facts, not faster recursion. 4 scoped commits on prolog-codeintel-verification (759e1f36..0cfe104a)
• SPEC-023 goal ACHIEVED: authoritative suite 612s wall (10:12.19, exit 0, kernel gRPC-verified) vs 905-910s quiescent baseline = 32.4% faster; 26859 passed / 0 failed / 0 errors / 0 skipped, 1333/1333 modules; batch phase 596s vs 869s (31.4%). Wired: janus fail-fast transport (791x), AST warm cache + incremental serialize, AST daemon + SessionStart hook + CLI delegation, parallel gates, Memgraph co-scheduling, replica canonical migration.
• (auto-closed by new session)

Handoff

• Key decisions: Never skip infra verification (user ruling)
• Blockers: None
• In-progress: CodeFunction carries no CALLS edges — possible KG-sync gap (backlogged)
• Important files: None
• Commands: None
• Warnings: Memgraph writer chain in flight: graph index -> wiki-rag-ingest -> final graph index (backgrounded); then run scripts/ingest_orchestrata_ultimate.py (full source re-ingest — changed src this session) as the LAST writer

Related

• Session log
• Session timeline
• Orchestrata Ultimate — project index
• Workflow