Orchestrata Ultimate › Plans › rtm
RTM — evidence-graded
Source: plans/rtm.md at 8a2cb42ad · last changed 2026-08-19 · first committed 2026-04-23 · 309 lines.
Prose pending.
Plan text
RTM — evidence-graded
Generated by
scripts/generate_rtm_evidence.py. Every row carries its proof class; a row upgrades ONLY when a test cites the requirement (and, under--verify, runs green). DECLARED rows are unverified decorator claims — measured 55% +/- 5% accurate (W6, 2026-07-09); do not treat them as ground truth.
Execution evidence this run: NO (static; CITED not executed)
Totals
| grade | requirements | share |
|---|---|---|
| CITED | 113 | 2.2% |
| DECLARED | 2695 | 52.3% |
| UNLINKED | 2335 | 45.3% |
| GHOST | 6 | 0.1% |
By spec
| spec | TESTED | FAILING | CITED | DECLARED | UNLINKED |
|---|---|---|---|---|---|
| 000-orchestrata-cli | 0 | 0 | 2 | 47 | 63 |
| 001-kernel | 0 | 0 | 37 | 210 | 58 |
| 002-postman | 0 | 0 | 0 | 221 | 236 |
| 003-intent | 0 | 0 | 2 | 279 | 91 |
| 004-cxp | 0 | 0 | 2 | 200 | 71 |
| 005-multi-agent | 0 | 0 | 0 | 25 | 23 |
| 006-session-management | 0 | 0 | 0 | 135 | 0 |
| 007-prolog-governance | 0 | 0 | 0 | 105 | 262 |
| 007-verification | 0 | 0 | 6 | 182 | 42 |
| 008-knowledge-graph | 0 | 0 | 0 | 68 | 4 |
| 009-volk-context-injection | 0 | 0 | 0 | 0 | 28 |
| 010-memory-engine | 0 | 0 | 0 | 25 | 0 |
| 011-provider-manager | 0 | 0 | 0 | 39 | 21 |
| 012-pm-volk | 0 | 0 | 1 | 42 | 7 |
| 013-ux-volk | 0 | 0 | 1 | 29 | 0 |
| 014-websocket-gateway | 0 | 0 | 2 | 58 | 142 |
| 015-plugin-sdk | 0 | 0 | 0 | 93 | 117 |
| 016-vizint-command-center | 0 | 0 | 0 | 38 | 256 |
| 017-calendar-integration | 0 | 0 | 0 | 23 | 47 |
| 018-fireflies-integration | 0 | 0 | 0 | 8 | 72 |
| 019-intent-impact-analyzer | 0 | 0 | 0 | 39 | 8 |
| 020-claude-code-harness | 0 | 0 | 0 | 8 | 232 |
| 021-opencode-integration | 0 | 0 | 0 | 1 | 294 |
| 022-tui-interface | 0 | 0 | 0 | 0 | 39 |
| 024-dependency-injection | 0 | 0 | 1 | 48 | 2 |
| 025-code-generation-drivers | 0 | 0 | 3 | 70 | 17 |
| 026-compliance | 0 | 0 | 0 | 35 | 0 |
| 027-junior-volk | 0 | 0 | 1 | 56 | 13 |
| 028-llm-client | 0 | 0 | 1 | 33 | 22 |
| 029-codedirector-volk | 0 | 0 | 0 | 73 | 2 |
| 030-architect-volk | 0 | 0 | 0 | 41 | 4 |
| 031-qa-volk | 0 | 0 | 6 | 36 | 13 |
| 032-concierge-volk | 0 | 0 | 2 | 46 | 6 |
| 050-cheetahclaws-integration | 0 | 0 | 0 | 4 | 1 |
| 063-repl-tool | 0 | 0 | 0 | 23 | 9 |
| 064-git-worktree-tools | 0 | 0 | 0 | 50 | 4 |
| 065-lsp-tool | 0 | 0 | 0 | 13 | 13 |
| 067-remote-trigger-tool | 0 | 0 | 9 | 14 | 0 |
| 068-schedule-cron-tool | 0 | 0 | 0 | 23 | 7 |
| 069-task-management-tools | 0 | 0 | 0 | 39 | 9 |
| 071-mcp-resource-tools | 0 | 0 | 0 | 16 | 32 |
| 072-tool-search-tool | 0 | 0 | 1 | 39 | 0 |
| 097-testing-standards | 0 | 0 | 0 | 0 | 11 |
| 102-volk-test-harness | 0 | 0 | 0 | 102 | 29 |
| 103-tool-calling-refactor | 0 | 0 | 8 | 15 | 2 |
| 104-openspec-format | 0 | 0 | 0 | 10 | 0 |
| 105-ontology-domain-model | 0 | 0 | 5 | 24 | 21 |
| 106-context-semantic-layer | 0 | 0 | 23 | 10 | 5 |
Evidence rows (TESTED / FAILING / CITED)
- REQ-000-053 [CITED]
orchestrata approve --thread <id>sends work_status=resolved
tests: tests/session/test_pipeline_cli_escalation.py - REQ-000-058 [CITED] Human must explicitly approve — no auto-approval
tests: tests/orchestrator/test_pipeline_escalation.py - REQ-001-001 [CITED] Volk can register with unique alias (e.g., U:Ox, U:PS)
tests: tests/scripts/test_ast_trace_audit.py - REQ-001-005 [CITED] Registration validates alias uniqueness
tests: tests/kernel/test_kernel_defect_regressions.py - REQ-001-018 [CITED] Discovery provides health status for each Volk
tests: tests/orchestrator/test_dispatch_heartbeat.py - REQ-001-US-002-001 [CITED] Route message based on TO field in CXP packet
tests: tests/scripts/test_ast_trace_audit.py - REQ-001-US-002-015 [CITED] Retry failed dispatches with exponential backoff
tests: tests/kernel/test_dispatcher_expanded.py - REQ-001-US-002-016 [CITED] Track dispatch attempts and failures
tests: tests/kernel/test_dispatcher_expanded.py - REQ-001-US-003-007 [CITED] Handle Prolog query timeouts
tests: tests/kernel/test_dispatcher_policy_audit.py - REQ-001-US-003-009 [CITED] Log all policy queries for audit
tests: tests/kernel/test_dispatcher_policy_audit.py - REQ-003-US-001-040 [CITED]
IntentVocabularysupports runtime-extensible keyword→category
tests: tests/orchestrator/test_intent_classifier.py, tests/orchestrator/test_intent_vocabulary.py - REQ-003-US-004-002 [CITED] Routing rules defined in Prolog
tests: tests/orchestrator/test_intent_router_prolog.py - REQ-001-US-004-001 [CITED] Create new session with unique session ID
tests: tests/session/test_kernel_session.py - REQ-001-US-004-002 [CITED] Initialize session context (facts, portfolio, carryover)
tests: tests/session/test_kernel_session.py - REQ-001-US-004-003 [CITED] Load session-specific Prolog rules
tests: tests/session/test_kernel_session.py - REQ-001-US-004-004 [CITED] Notify all Volks of session start
tests: tests/session/test_kernel_session.py - REQ-001-US-004-005 [CITED] Persist session metadata to Redis
tests: tests/session/test_kernel_session.py - REQ-001-US-004-006 [CITED] Support session templates (pre-configured sessions)
tests: tests/session/test_kernel_session.py - REQ-001-US-004-007 [CITED] Validate session configuration
tests: tests/session/test_kernel_session.py - REQ-001-US-004-008 [CITED] Handle concurrent session initialization
tests: tests/session/test_kernel_session.py - REQ-001-US-004-009 [CITED] Set session resource limits
tests: tests/session/test_kernel_session.py - REQ-001-US-004-010 [CITED] Log session initialization events
tests: tests/session/test_kernel_session.py - REQ-001-US-004-011 [CITED] Maintain session-specific facts
tests: tests/session/test_kernel_session.py - REQ-001-US-004-012 [CITED] Maintain session portfolio (artifacts)
tests: tests/session/test_kernel_session.py - REQ-001-US-004-013 [CITED] Maintain session carryover (todos)
tests: tests/session/test_kernel_session.py - REQ-001-US-004-014 [CITED] Isolate session state from other sessions
tests: tests/session/test_kernel_session.py - REQ-001-US-004-015 [CITED] Support session context queries
tests: tests/session/test_kernel_session.py - REQ-001-US-004-016 [CITED] Support session context updates
tests: tests/session/test_kernel_session.py - REQ-001-US-004-017 [CITED] Validate context schema
tests: tests/session/test_kernel_session.py - REQ-001-US-004-018 [CITED] Persist context changes to Redis
tests: tests/session/test_kernel_session.py - REQ-001-US-004-019 [CITED] Handle context merge conflicts
tests: tests/session/test_kernel_session.py - REQ-001-US-004-020 [CITED] Provide context versioning
tests: tests/session/test_kernel_session.py - REQ-001-US-004-021 [CITED] Gracefully terminate session
tests: tests/session/test_kernel_session.py - REQ-001-US-004-022 [CITED] Archive session artifacts
tests: tests/session/test_kernel_session.py - REQ-001-US-004-023 [CITED] Clean up session resources
tests: tests/session/test_kernel_session.py - REQ-001-US-004-024 [CITED] Notify all Volks of session end
tests: tests/session/test_kernel_session.py - REQ-001-US-004-025 [CITED] Generate session summary report
tests: tests/session/test_kernel_session.py - REQ-001-US-004-026 [CITED] Persist final session state
tests: tests/session/test_kernel_session.py - REQ-001-US-004-027 [CITED] Handle forced session termination
tests: tests/session/test_kernel_session.py - REQ-001-US-004-028 [CITED] Transfer carryover to next session
tests: tests/session/test_kernel_session.py - REQ-004-SUCC-005 [CITED] Unit tests passing for all user stories
tests: tests/gateway/test_cli_behavioral.py - REQ-004-SUCC-006 [CITED] Integration tests passing with Postman and Volks
tests: tests/gateway/test_cli_behavioral.py - REQ-001-US-005-003 [CITED] Detect resource conflicts
tests: tests/governance/test_escalation_handler.py - REQ-009-US-005-001 [CITED] Support Claude Opus 4 for verification
tests: tests/verification/agents/test_supervisor_agent.py - REQ-009-US-005-005 [CITED] Select appropriate model based on task
tests: tests/verification/agents/test_verification_agent.py - REQ-009-US-005-009 [CITED] Support model fallback chain
tests: tests/verification/models/test_base_model.py - REQ-009-US-007-009 [CITED] Request additional proof if needed
tests: tests/verification/orchestration/test_coordinator.py - REQ-009-US-009-001 [CITED] Receive intents from Intent Broker
tests: tests/verification/session/test_verification_session.py - REQ-009-US-009-004 [CITED] Return verification results to Intent Broker
tests: tests/verification/orchestration/test_coordinator.py - REQ-012-005 [CITED] Unrecognised subjects are returned as an error CXPPacket without raising an exception.
tests: tests/generated_recovery/test_agents_volks_pm_volk.py - REQ-013-030 [CITED] constructs a CXP addressed to U:Ar with subject "ux_complete" containing the field plus all preserved payload context fields
tests: tests/agents/volks/ux/test_volk.py - REQ-014-101 [CITED] Prolog queried for
gateway_access(ClientId, Scope)on auth
tests: tests/gateway/test_cli_behavioral.py - REQ-014-102 [CITED] Scoped access enforced: read_only, read_write, admin
tests: tests/gateway/test_cli_behavioral.py - REQ-024-034 [CITED] build() validates that all registrations are internally consistent (no duplicate interfaces, valid implementations) and raises ValueError on the first violation
tests: tests/test_di.py - REQ-025-037 [CITED] Each path in
filesappended as separate--file <path>argument
tests: tests/drivers/test_aider_driver.py - REQ-025-038 [CITED] Timeout of 300 seconds enforced
tests: tests/drivers/test_aider_driver.py - REQ-025-040 [CITED] Binary not found returns
(False, "Aider CLI not found")
tests: tests/drivers/test_aider_driver.py - REQ-027-031 [CITED]
CODING_STANDARDSappended toDESIGN_REVIEW_PROMPTwith brace-escaping
tests: tests/agents/volks/junior/test_junior_handoff.py - REQ-028-021 [CITED] A sentinel
defaultkey inVolkConfigmap applies to any unrecognisedvolk_id
tests: tests/drivers/test_deepseek_coder_driver.py, tests/llm/test_config.py, tests/test_llm.py - REQ-031-039 [CITED] LLM review call uses
temperature=0.0,volk="U:Qa", and injectsself._graph_context
tests: tests/agents/volks/qa/test_llm_review.py - REQ-031-040 [CITED]
_parse_review_verdict()extracts "PASS" or "FAIL" from the REVIEW_VERDICT: line in the LLM response
tests: tests/agents/volks/qa/test_llm_review.py - REQ-031-041 [CITED]
_parse_review_issues()extracts actionable issues from the ISSUES: block when verdict is FAIL
tests: tests/agents/volks/qa/test_llm_review.py - REQ-031-042 [CITED] FAIL verdict triggers
_bounce_for_fix()to U:Cd with parsed issues
tests: tests/agents/volks/qa/test_llm_review.py - REQ-031-043 [CITED] Review runs on every cycle, including when packet carries
from_test_fix=True(G226)
tests: tests/agents/volks/qa/test_llm_review.py, tests/agents/volks/qa/test_volk_extended.py - REQ-031-053 [CITED]
_retry_without_mocks()re-runs LLM generation with anti-mock instruction when mock usage is detected
tests: tests/agents/volks/qa/test_testgen.py - REQ-032-014 [CITED] Packets with unrecognised subjects are rejected with a logged error
tests: tests/agents/volks/concierge/test_volk.py - REQ-032-041 [CITED] UI detection scans combined conversation_history text for UI-related keywords
tests: tests/agents/volks/concierge/test_volk.py - REQ-067-001 [CITED] Execute HTTP requests (GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS) to a configured URL with configurable headers/body.
tests: tests/tools/test_remote_trigger_tool.py - REQ-067-002 [CITED] API Key authentication, placed in a header or a query parameter (
in: header|query).
tests: tests/tools/test_remote_trigger_tool.py - REQ-067-008 [CITED] Configurable max attempts, exponential backoff (
initial_delay*backoff_factor** attempt, capped atmax_delay), and configurable retryable status codes.
tests: tests/tools/test_remote_trigger_tool.py - REQ-067-011 [CITED] Validate against expected HTTP status codes (defaults to 2xx when unset).
tests: tests/tools/test_remote_trigger_tool.py - REQ-067-015 [CITED] Configurable per-request timeout with a clear timeout error message ("Request timeout after {n} seconds").
tests: tests/tools/test_remote_trigger_tool.py - REQ-067-018 [CITED] Parses JSON responses (content-type aware, falls back to raw text) and extracts dotted-path fields (
extract_fields) into the response. Response-format convers
tests: tests/tools/test_remote_trigger_tool.py - REQ-067-020 [CITED] Blocks outbound requests to loopback/private/link-local/reserved addresses and known metadata hostnames/IPs (single-resolution check via
socket.getaddrinfo),
tests: tests/tools/test_remote_trigger_tool.py - REQ-067-021 [CITED]
execute_simple(url, method, headers, body, auth, timeout, env)simplified interface.
tests: tests/tools/test_remote_trigger_tool.py - REQ-067-023 [CITED] ERR-00N conditions (invalid URL, auth failure, forbidden, not found, rate limited, server error, timeout, connection error, SSL error, invalid signature, retry
tests: tests/tools/test_remote_trigger_tool.py - REQ-072-201 [CITED] SHALL sanitize all query inputs to prevent injection attacks
tests: tests/tools/test_search.py - REQ-103-017 [CITED] The loop sends the system prompt + task +
TOOL_SCHEMAS, and while the model returnstool_calls, executes each viadispatch, appends onetoolmessage per
tests: tests/refactor/test_agent.py - REQ-103-018 [CITED] The loop terminates when the model returns content with no tool calls (
status="completed"), aftermax_iterations(status="max_iterations"), or on a fatal
tests: tests/refactor/test_agent.py - REQ-103-019 [CITED] The full assistant/tool message history is preserved across turns so the model retains continuity
tests: tests/refactor/test_agent.py - REQ-103-020 [CITED] The refactoring system prompt instructs the model to OBSERVE before MUTATING (read/outline/grep first), make MINIMAL surgical edits via
edit_file, preserve be
tests: tests/refactor/test_agent.py - REQ-103-021 [CITED] The prompt forbids fabricating file contents and instructs the model to copy exact snippets from
read_fileforedit_file
tests: tests/refactor/test_agent.py - REQ-103-022 [CITED]
refactor(task: str, root: str | Path, model: str = "deepseek-v4-pro", max_iterations: int = 25) -> RefactorResultruns the tool-use loop and returns a `Refact
tests: tests/refactor/test_agent.py - REQ-103-023 [CITED]
RefactorResultrecordsfiles_changed,iterations, the orderedtool_callstrace,verified, andstatus
tests: tests/refactor/test_agent.py - REQ-103-025 [CITED] When intent classification yields
IntentType.REFACTOR, the CodeDirector path MAY dispatch to this refactor capability instead of greenfield scaffolding — a de
tests: tests/orchestrator/test_intent_broker_refactor.py - REQ-105-017 [CITED]
owl:minCardinality >= 1(orowl:someValuesFrom) →Field/Relationship.required=True.
tests: tests/domaingen/test_reader.py - REQ-105-019 [CITED]
rdfs:label→ human name (fallback: IRI local name);rdfs:comment→ docstring.
tests: tests/domaingen/test_domaingen.py - REQ-105-038 [CITED] A separate transaction-time-only ("as-recorded") and valid-time-only query mode is supported.
tests: tests/domaingen/test_domaingen.py - REQ-105-042 [CITED] Stardog connection failure raises
RepositoryError/StardogConnectionError(bothBaseError); code generation itself is offline and unaffected (ERR-105-005).
tests: tests/domaingen/test_domaingen.py - REQ-105-049 [CITED]
generate_shaclemits ash:NodeShapeper entity (sh:targetClass); fields → property shapes withsh:datatype+sh:maxCount 1(+sh:minCount 1when req
tests: tests/domaingen/test_domaingen.py - REQ-106-001 [CITED] Code nodes use labels Module/Class/Function with id
<relpath>::<qualname>; methods are Function nodes carrying class_id
tests: tests/knowledge_graph/test_canonical_schema.py - REQ-106-002 [CITED] Requirement nodes key on the literal requirement id string; TRACES_TO targets RequirementItem
tests: tests/knowledge_graph/test_canonical_schema.py - REQ-106-005 [CITED] schema.py get_all_ddl() is the single constraint source and has an apply CLI
tests: tests/knowledge_graph/test_canonical_schema.py - REQ-106-006 [CITED] A conformance gate flags any MERGE using non-canonical label/key/rel and runs in CI
tests: tests/knowledge_graph/test_canonical_schema.py - REQ-106-011 [CITED] Task-type classification reuses the existing deterministic parsers (intent/work_type/SUBJECT_TASK_TYPE tables)
tests: tests/knowledge_graph/context_pack/test_frame.py - REQ-106-012 [CITED] Entity linking matches exact and normalized node names against the canonical graph
tests: tests/knowledge_graph/context_pack/test_frame.py - REQ-106-013 [CITED] Ambiguous entity matches are retained with recorded ambiguity, never dropped or guessed
tests: tests/knowledge_graph/context_pack/test_frame.py - REQ-106-014 [CITED] Identical request text yields an identical RequestFrame (pure function of input + graph names)
tests: tests/knowledge_graph/context_pack/test_frame.py - REQ-106-021 [CITED] Expansion is k-hop bounded with a default of 2 and per-template overrides
tests: tests/knowledge_graph/context_pack/test_selector.py - REQ-106-022 [CITED] Ranking uses only graph signals: hop distance, trace degree, TESTED_BY presence, recency, ambiguity penalty
tests: tests/knowledge_graph/context_pack/test_selector.py, tests/knowledge_graph/test_context_injector_label_regression.py, tests/knowledge_graph/test_context_injector_selector_wiring.py, tests/knowledge_graph/test_context_injector_task_types.py - REQ-106-023 [CITED] Rank ties break lexicographically on node id for reproducibility
tests: tests/knowledge_graph/context_pack/test_relevance.py, tests/knowledge_graph/context_pack/test_selector.py - REQ-106-025 [CITED] The six SPEC-009 task-type handlers are expressed as selection templates without losing their traced REQ coverage
tests: tests/knowledge_graph/context_pack/test_selector.py - REQ-106-030 [CITED] Packing is greedy by rank under an explicit token budget that is never exceeded
tests: tests/knowledge_graph/context_pack/test_packer.py - REQ-106-033 [CITED] Every pack section carries provenance: node id, selecting template, rank, token cost
tests: tests/knowledge_graph/context_pack/test_packer.py - REQ-106-034 [CITED] Same frame + same graph state produces a byte-identical pack body
tests: tests/knowledge_graph/context_pack/test_packer.py - REQ-106-035 [CITED] A trimmed pack (sections deleted) re-validates and injects without regeneration
tests: tests/knowledge_graph/context_pack/test_packer.py - REQ-106-040 [CITED] ContextInjector.inject_for_task consumes the selector/packer instead of ad-hoc LIMIT queries
tests: tests/knowledge_graph/test_context_injector_task_types.py - REQ-106-042 [CITED] Each injection records budget, spend and section list for measurement
tests: tests/knowledge_graph/test_context_injector_selector_wiring.py - REQ-106-043 [CITED] KG-unavailable degrades to an explicit empty pack with reason, never a silent partial
tests: tests/knowledge_graph/context_pack/test_packer.py, tests/scripts/test_context_pack_cli.py - REQ-106-061 [CITED] kg_relevant/1 and kg_relevance_tier/2 are deduced by task-type rules — a linked root entity is always core; a candidate that shares_requirement (transitive over
tests: tests/knowledge_graph/context_pack/test_relevance.py - REQ-106-062 [CITED] Tiers feed FR-003's score as a discrete bonus (core >> related >> peripheral), applied after the linear score, with the FR-003 lexicographic node-id tie-break u
tests: tests/knowledge_graph/context_pack/test_relevance.py - REQ-106-063 [CITED] When no Prolog bridge is wired every node is peripheral (zero bonus) and ranking is byte-identical to the FR-003-only path — FR-006 is strictly opt-in
tests: tests/knowledge_graph/context_pack/test_relevance.py - REQ-106-064 [CITED] Projected facts are request-scoped and retracted after each refinement (no cross-request fact leakage); the ruleset is consulted once and reused
tests: tests/knowledge_graph/context_pack/test_relevance.py
GHOST ids (referenced but defined in no spec)
- REQ-001 (first ref: tests/session/test_kg_sync_extended.py:124)
- REQ-001- (first ref: tests/scripts/test_ast_trace_audit.py:169)
- REQ-024-044 (first ref: tests/test_di.py:934)
- REQ-042-002 (first ref: tests/scripts/test_generate_rtm_evidence.py:47)
- REQ-1 (first ref: tests/agents/test_spec_writer_extended.py:586)
- REQ-999-999 (first ref: tests/scripts/test_ast_check_traces.py:102)