Knowledge Wiki Orchestrata Ultimate

Orchestrata Ultimate › Plans › context-injection-phase2-plan

Phase 2 KG Enhancements — Comprehensive Execution Plan

Source: plans/context-injection-phase2-plan.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-14 · 323 lines.

Prose pending.

Plan text

Phase 2 KG Enhancements — Comprehensive Execution Plan

Continuation of plans/context-injection-tracked-plan.md. This plan closes the seven pending Phase 2 items (2.2–2.8) that enrich the graph context injected into volk prompts and the feedback loop around it.

Status: planning
Target completion: one commit per numbered task, tests + health_check.py green before moving on.

0. Principles and constraints

  1. Real infrastructure only. Every test must exercise real Neo4j/Memgraph or degrade gracefully; no mocks, stubs, or monkeypatch.
  2. No schema guesswork. New node/edge types or property shapes must be added to knowledge_graph/canonical_schema.py (identity + relationships) and knowledge_graph/schema.py (constraints/indexes) before any writer/query uses them.
  3. Project-scoped. All new KG queries respect project_id when the ContextInjector was configured with one; default behaviour remains project-agnostic.
  4. Backward compatibility. Existing graph_context prompt shape must not break; new sections are additive.
  5. No pytest-xdist. Use scripts/run_tests.py or the streaming runner.
  6. Traceability. New public functions/classes get @traces with real REQ ids from the spec they implement (SPEC-008 / SPEC-106 / SPEC-107).

1. Proposed execution order

Order Task Why this order
2.2 Recent-changes section Self-contained; no dependencies on other Phase 2 items.
2.3 Similar-patterns section Self-contained; can be done in parallel with 2.2.
2.5 Semantic-search bridge Reusable bridge component; later items can call it.
2.8 Unify Requirement/RequirementItem Cleans the data model used by 2.4 and 2.6.
2.4 Requirement closure Builds on unified requirement identity.
2.7 Runtime Tier-H ingestion Wires existing RuntimeSignalWriter into live pipeline paths.
2.6 Validator feedback loop Touches the LLM call path; do last so the richer context from 2.2–2.5 is available to retry with.

2. Task 2.2 — Recent-changes section

Current state

Goal

Populate GraphContext.recent_changes with the most recent modifications to the target files and their direct neighbours, then render them in GraphContext.to_prompt().

Design

  1. Add a _query_recent_changes(file_paths, context) helper to ContextInjector.
  2. Query options (try in order, stop when results found): a. (:File)-[:CHANGED_IN]->(:Change {timestamp, description}) if Change nodes exist. b. (:File)-[:NEXT_VERSION]->(:File) timestamp diff. c. Fallback: use the file node’s own updated_at / synced_at property.
  3. Sort by timestamp desc, cap at 5 entries.
  4. Each entry: {"file": path, "change": description, "timestamp": iso, "author": "..."}.
  5. Render in _format_recent_changes_section() (already exists; just needs data).

Files to touch

Acceptance criteria

Tests

3. Task 2.3 — Similar-patterns section

Current state

Goal

Populate GraphContext.similar_patterns with files/classes that implement the same protocols or inherit from the same bases as the target files, so volks know where to look for analogous code.

Design

  1. Add _query_similar_patterns(file_paths, context) helper.
  2. For each target file, find its classes and their base classes / implemented protocols.
  3. Query Neo4j for other File nodes whose classes share at least one base/protocol.
  4. Rank by number of shared bases and then by path similarity.
  5. Cap at 5 files.
  6. Render as a new ## Similar Patterns section in to_prompt().

Files to touch

Acceptance criteria

Tests

4. Task 2.5 — Semantic-search bridge

Current state

Goal

Extract the Memgraph-chunk → Neo4j-root logic into a reusable bridge and optionally make it available to ContextInjector so GraphContext can include semantic-search results.

Design

  1. Create src/orchestrata_ultimate/knowledge_graph/semantic_bridge.py with:
    • search_semantic_roots(query: str, target_files: list[str], *, project_id, config, limit) -> list[dict[str, Any]]
    • Returns Neo4j File records + matching chunk snippets.
  2. Refactor context/providers/semantic_context.py to import and use semantic_bridge.search_semantic_roots.
  3. Optionally add _query_semantic_bridge(file_paths, context) in ContextInjector to append semantic roots to context.similar_patterns or a new context.semantic_files field.

Files to touch

Acceptance criteria

Tests

5. Task 2.8 — Unify Requirement/RequirementItem

Current state

Goal

Create a canonical requirement-resolution helper so every KG query treats Requirement and RequirementItem as one logical requirement chain.

Design

  1. Add knowledge_graph/requirement_resolution.py:
    • resolve_requirement_chain(ri_ids: list[str], repo, project_id) -> list[dict] — returns merged Requirement+Item records with stable id, description, specs.
  2. Update ContextInjector._query_semantic_context to use the helper.
  3. Update context_pack/relevance.py parent-requirement query to use the same helper (or at least the same canonical keys).
  4. Add a RequirementChain dataclass to avoid dict drift.

Files to touch

Acceptance criteria

Tests

6. Task 2.4 — Requirement closure

Current state

Goal

Walk the TRACES_TO / HAS_ITEM / REQUIRES hierarchy so volks see not only the exact requirement a symbol traces to, but also its siblings, parents, and dependencies.

Design

  1. Extend requirement_resolution.py with:
    • closure_requirement_chain(seed_ids, repo, project_id, max_depth=2) -> list[dict]
  2. Use the closure in ContextInjector._query_semantic_context.
  3. Cap total returned requirements at 20 to avoid prompt bloat.
  4. Mark each requirement in the rendered output with its distance from the seed (direct, parent, dependency).

Files to touch

Acceptance criteria

Tests

7. Task 2.7 — Runtime Tier-H ingestion

Current state

Goal

Wire runtime signal writing into the live pipeline so messages, decisions, and tool calls become queryable context for DEBUG/retry scenarios.

Design

  1. Messages: In orchestrator/pipeline_runner.py and kernel/pipeline.py, after a CXP packet is dispatched/received, call runtime_writer.write_message() with sender/recipient/subject and any file/entity mentions extracted from the payload.
  2. Decisions: PipelineKGMixin.sync_decisions_to_kg already writes durable Decision nodes; also call runtime_writer.write_decision() (or unify the two) so the decision appears as a Tier-H signal.
  3. Tool calls: Add a hook in BaseAgent._call_llm_with_tools / tool execution to call runtime_writer.write_tool_call() with the tool name, arguments, result, and mentioned files/entities.
  4. Entity extraction helper: Add knowledge_graph/signal_extraction.py to extract file paths and code entity ids from a payload/text using regex and the AST index, reusable by all three writers.
  5. Ensure writes are best-effort and never raise.

Files to touch

Acceptance criteria

Tests

8. Task 2.6 — Validator feedback loop

Current state

Goal

When validation fails because core entities are missing from the response, automatically expand the context (larger k-hop, include TESTED_BY / TRACES_TO neighbours) and retry the LLM call, up to a limit.

Design

  1. Add knowledge_graph/context_validator_loop.py:
    • ValidatorLoop(injector, validator, max_retries=2)
    • run(payload, call_llm, *, task_type) calls LLM, validates, and on missing citations expands context via injector.inject(..., expansion=...) and retries.
  2. Add an expansion: ContextExpansion parameter to ContextInjector.inject / inject_for_task:
    • k_hop increase
    • include TESTED_BY neighbours
    • include requirement closure
  3. Wire the loop into BaseAgent._call_llm as an opt-in path for subjects in INJECT_SUBJECTS.
  4. Keep the default ContextInjector.inject() unchanged so existing callers are unaffected.

Files to touch

Acceptance criteria

Tests

9. Cross-cutting work

Schema

Performance

Documentation

Verification per task

  1. uv run pytest tests/<relevant>/ -q passes.
  2. python scripts/health_check.py passes.
  3. uv run ast-sync regenerates AST files if source layout changed.
  4. git add -A && git commit -m "P2.X ..." && git push origin master.

10. Definition of done for Phase 2