Orchestrata Ultimate / Specs / 026-compliance
026-compliance: Compliance Volk (U:Cp)
ComplianceVolk (alias U:Cp, role "Compliance") is a deterministic volk that runs after U:Qa completes. It executes a 7-step pipeline that scans @traces decorators from source and test code, loads requirements from the Neo4j knowledge graph, generates a Requirements Traceability Matrix (RTM), syncs code nodes back to the KG, checks design-vs-code compliance, persists the RTM to disk and the KG, and produces a compliance report with coverage % and compliance % metrics.
At a glance
| Item | Value |
|---|---|
| Spec directory | specs/026-compliance/ |
| Status line in the spec | Implemented |
| Requirement IDs defined | 35 |
With a @traces citation in src/ |
35 (100.0%) |
With a @traces citation in tests/ |
0 (0.0%) |
| Tracked files | 6 |
| RTM page | 026-compliance |
Reading this spec
The Compliance Volk (alias U:Cp, role Compliance) is a deterministic agent that runs after the QA Volk completes. It executes a 7-step pipeline that scans @traces decorators in source and test code, loads requirements from the Neo4j knowledge graph, generates a Requirements Traceability Matrix (RTM), syncs code nodes back to the graph and checks the design against the code. It exists so that traceability between requirements and implementation is verified automatically after every code generation cycle, design drift is detected before delivery, and the RTM is saved for audit without manual effort. The step is deterministic on purpose: no LLM inference enters governance decisions.
The scope is five user stories: automated RTM generation, design-versus-code compliance check, code sync to the knowledge graph and RTM persistence to the graph (priority 1), and a human-readable compliance report (priority 2). Requirement IDs use the REQ-026- prefix. The spec has a terminology table covering RTM, TraceInfo, RTMReport, ValidationReport, ComplianceResult and RTMSnapshot.
The design decisions are that ComplianceVolk extends BaseVolk, routes CXP packets on the subject compliance to an internal pipeline, and keeps all heavy logic in _pipeline.py as pure functions with no shared state. The graph connection is created lazily, and failure is non-fatal: the pipeline degrades gracefully. The response is a CXP packet to the human with coverage and compliance metrics. The RTM is saved under the project docs as JSON and markdown and as an RTMSnapshot node in Neo4j.
In the code, the work is in orchestrata_ultimate/agents/volks/compliance/_pipeline and orchestrata_ultimate/agents/volks/compliance/volk, with orchestrata_ultimate/core/traceability_utils behind them. The per-requirement view is on the RTM page.
The spec's status line reads Implemented. It tracks 6 files and defines 35 requirement IDs. The full text follows below.
Measured coverage is 35 of 35 requirements cited by at least one @traces in src/ (100.0%), spread over 3 modules; the module citing the most of them is orchestrata_ultimate/agents/volks/compliance/_pipeline (24 requirements). In tests/, 0 requirements are cited by a test-side @traces.
Coverage gaps: none on the citation measure, every requirement has at least one citing src/ symbol. Test-side citations are thinner than code-side ones (0 against 35).
Requirements
Checkbox is the state recorded in the requirements checklist; Traced by is the count of src/ symbols whose @traces names the ID.
| ID | Done | Requirement | Traced by |
|---|---|---|---|
REQ-026-001 |
no | ComplianceVolk.process() routes packets with subject compliance_check to _handle_compliance(); all other subjects return None |
1 |
REQ-026-002 |
no | _handle_compliance() calls _extract_packet_fields(), _run_pipeline(), and _build_response() in sequence |
1 |
REQ-026-003 |
no | _run_pipeline() executes all 7 pipeline steps in order: scan_traces → load_requirements → generate_rtm → sync_code → check_compliance → persist_rtm → _calculate_results |
1 |
REQ-026-004 |
no | WorkStatus transitions from PENDING to IN_PROGRESS on packet receipt and to RESOLVED on pipeline completion | 3 |
REQ-026-005 |
no | The volk summary string is updated with PASS/FAIL status plus coverage and compliance percentages on completion | 2 |
REQ-026-006 |
no | scan_traces(src_dir, tests_dir) scans @traces decorators in Python files under src_dir and tests_dir using orchestrata.core.traceability_utils.scan_traces |
9 |
REQ-026-007 |
no | Scanning is skipped for any directory that does not exist (os.path.isdir check); no exception is raised | 1 |
REQ-026-008 |
no | Trace records from both src_dir and tests_dir are merged into a single list | 1 |
REQ-026-009 |
no | If scan raises an exception, an empty list is returned and the error is logged at WARNING level | 1 |
REQ-026-010 |
no | load_requirements(kg_store, task_summary) fetches requirements from the KG via kg_store.get_requirements(slug) |
2 |
REQ-026-011 |
no | If kg_store is None, load_requirements returns an empty list immediately without attempting a KG call | 1 |
REQ-026-012 |
no | KG requirement dicts are converted to Requirement dataclass instances with id, description, story_id, and spec_id fields | 2 |
REQ-026-013 |
no | If the KG call raises an exception, an empty list is returned and the error is logged at WARNING level | 1 |
REQ-026-014 |
no | generate_rtm(traces, requirements, task_summary) calls orchestrata.core.traceability_utils.generate_rtm with trace list, requirement list, and project slug |
3 |
REQ-026-015 |
no | If generation raises an exception, None is returned and the error is logged at WARNING level | 1 |
REQ-026-016 |
no | On success, RTM validation metrics (verified, missing, orphan, coverage %) are logged at INFO level | 1 |
REQ-026-017 |
no | sync_code(kg_store, task_summary, trace_records) converts each TraceInfo to a dict via to_dict() if available, then calls kg_store.sync_code(slug, dicts) |
1 |
REQ-026-018 |
no | If kg_store is None, sync_code returns immediately without action | 1 |
REQ-026-019 |
no | If the KG call raises an exception, the error is logged at WARNING level and execution continues | 1 |
REQ-026-020 |
no | check_compliance(kg_store, task_summary) calls kg_store.check_compliance(slug) and returns the ComplianceResult |
1 |
REQ-026-021 |
no | If kg_store is None, check_compliance returns None immediately | 1 |
REQ-026-022 |
no | On success, compliance metrics (compliance %, missing count, extra count) are logged at INFO level | 1 |
REQ-026-023 |
no | If the KG call raises an exception, None is returned and the error is logged at WARNING level | 1 |
REQ-026-024 |
no | persist_rtm(kg_store, rtm_report, session_id, project_dir) calls _save_rtm_to_disk and _save_rtm_to_kg in sequence |
1 |
REQ-026-025 |
no | If rtm_report is None, persist_rtm returns an empty dict immediately | 1 |
REQ-026-026 |
no | _save_rtm_to_disk calls orchestrata.core.traceability_utils.save_rtm targeting <project_dir>/docs/ |
3 |
REQ-026-027 |
no | _save_rtm_to_kg creates an RTMSnapshot with unique ID ("rtm-" + uuid4().hex[:8]) and calls kg_store.close_session() |
1 |
REQ-026-028 |
no | If session_id is empty or kg_store is None, the KG snapshot write is skipped without error | 1 |
REQ-026-029 |
no | _calculate_results extracts coverage_pct from rtm_report.validation.coverage_percent and compliance_pct from compliance_result.compliance_pct; both default to 0.0 if unavailable |
3 |
REQ-026-030 |
no | passed is True when coverage_pct >= compliance_threshold (default 0.0) |
1 |
REQ-026-031 |
no | build_report_lines() produces a list of strings with RTM Coverage, Design Compliance, RTM Files, and Result sections |
4 |
REQ-026-032 |
no | The completed report string is appended to self.output_artifacts | 1 |
REQ-026-033 |
no | _init_kg() imports and constructs KGDesignStore only on first call; subsequent calls are no-ops when kg_store is already set |
1 |
REQ-026-034 |
no | If KGDesignStore construction raises any exception, kg_store is set to None and the error is logged at WARNING level | 1 |
REQ-026-035 |
no | _task_slug(task_summary) converts task_summary to lowercase, replaces all non-word/non-hyphen characters with hyphens, strips leading/trailing hyphens, and truncates to 50 characters |
1 |
Code that traces this spec
| Module | Symbols | Requirements |
|---|---|---|
orchestrata_ultimate/agents/volks/compliance/_pipeline |
10 | 24 |
orchestrata_ultimate/agents/volks/compliance/volk |
15 | 11 |
orchestrata_ultimate/core/traceability_utils |
12 | 3 |
Checklists
checklists/constitutional.md
Compliance Volk Constitutional Checklist
Determinism and Governance Integrity
- [ ] CONST-026-001: ComplianceVolk must make zero LLM API calls; all compliance decisions are deterministic based on code traces and KG state alone
- [ ] CONST-026-002: Given identical inputs and KG state, the compliance pipeline must produce byte-for-byte reproducible output
- [ ] CONST-026-003: Compliance threshold enforcement must not be bypassed; the passed flag is computed strictly from coverage_pct >= compliance_threshold
- [ ] CONST-026-004: RTM entries must accurately reflect only the trace annotations present in code; no fabrication or inference of coverage
Safety and Non-Interference
- [ ] CONST-026-005: ComplianceVolk must not write to source or test files; it is read-only with respect to the project under review
- [ ] CONST-026-006: KG writes (sync_code, RTMSnapshot, close_session) must use MERGE semantics to be idempotent; re-running must not duplicate nodes
- [ ] CONST-026-007: Disk writes (RTM JSON and markdown) must target only the project docs/ directory; no writes outside the project_dir tree
- [ ] CONST-026-008: Pipeline failures must never leave partial state that corrupts subsequent runs; all steps catch exceptions and return safe defaults
Transparency and Auditability
- [ ] CONST-026-009: Every compliance run must log its outcome (PASS/FAIL, coverage %, compliance %) at INFO level for audit trail
- [ ] CONST-026-010: Every pipeline step failure must be logged at WARNING level with the exception message for diagnosis
- [ ] CONST-026-011: RTM snapshots persisted to Neo4j must include verified_count, missing_count, orphan_count, and coverage_pct for historical queries
- [ ] CONST-026-012: The compliance report artifact appended to output_artifacts must include all three sections: RTM Coverage, Design Compliance, and Result
Graceful Degradation
- [ ] CONST-026-013: KG unavailability must not prevent RTM generation; trace scanning and RTM construction must complete using only the filesystem
- [ ] CONST-026-014: Missing source or test directories must not cause exceptions; the pipeline must continue with empty trace lists
- [ ] CONST-026-015: A None rtm_report or None compliance_result must produce a valid (though minimal) compliance report rather than crashing
checklists/requirements.md
The requirement list above comes from this file; 0 of 35 checkbox items are ticked.
checklists/testing.md
Compliance Volk Testing Checklist
Volk Instantiation
- [ ] TEST-026-001: Verify ComplianceVolk.alias == "U:Cp"
- [ ] TEST-026-002: Verify ComplianceVolk.role == "Compliance"
- [ ] TEST-026-003: Verify default compliance_threshold == 0.0
- [ ] TEST-026-004: Verify default project_dir is empty string
- [ ] TEST-026-005: Verify initial WorkStatus is PENDING
- [ ] TEST-026-006: Verify kg_store is None on fresh instantiation
- [ ] TEST-026-007: Verify output_artifacts is an empty list on fresh instantiation
CXP Packet Routing
- [ ] TEST-026-008: Process packet with subject "compliance_check" and valid payload returns a CXPPacket (not None)
- [ ] TEST-026-009: Process packet with unknown subject returns None
- [ ] TEST-026-010: Process packet with subject "compliance_check" returns packet with subject "compliance_complete"
- [ ] TEST-026-011: Process packet with subject "compliance_check" returns packet with from_agent "U:Cp"
- [ ] TEST-026-012: Process packet with subject "compliance_check" and empty project_dir completes without raising exception
Pipeline — Trace Scanning
- [ ] TEST-026-013: scan_traces with non-existent src_dir and tests_dir returns empty list
- [ ] TEST-026-014: scan_traces with valid src_dir containing @traces-annotated files returns non-empty list of TraceInfo objects
- [ ] TEST-026-015: scan_traces with only tests_dir existing returns traces from tests only
- [ ] TEST-026-016: scan_traces merges src and test traces into a single list
Pipeline — Requirements Loading
- [ ] TEST-026-017: load_requirements with kg_store=None returns empty list
- [ ] TEST-026-018: load_requirements with kg_store that raises exception returns empty list
- [ ] TEST-026-019: load_requirements with kg_store returning empty list returns empty list
- [ ] TEST-026-020: load_requirements converts KG dicts to Requirement instances with correct id and description fields
Pipeline — RTM Generation
- [ ] TEST-026-021: generate_rtm with empty trace_records and empty requirements returns RTMReport with 100.0 coverage_percent (no requirements to miss)
- [ ] TEST-026-022: generate_rtm with traces covering all requirements returns validation.coverage_percent == 100.0
- [ ] TEST-026-023: generate_rtm with traces covering half the requirements returns validation.coverage_percent == 50.0
- [ ] TEST-026-024: generate_rtm when underlying function raises exception returns None
Pipeline — Code Sync
- [ ] TEST-026-025: sync_code with kg_store=None returns without error
- [ ] TEST-026-026: sync_code with kg_store that raises exception logs warning and continues without raising
- [ ] TEST-026-027: sync_code calls to_dict() on TraceInfo objects before passing to kg_store
Pipeline — Compliance Check
- [ ] TEST-026-028: check_compliance with kg_store=None returns None
- [ ] TEST-026-029: check_compliance with kg_store that raises exception returns None
- [ ] TEST-026-030: check_compliance with valid kg_store returns ComplianceResult with compliance_pct, missing_classes, and extra_classes
Pipeline — RTM Persistence
- [ ] TEST-026-031: persist_rtm with rtm_report=None returns empty dict
- [ ] TEST-026-032: persist_rtm with valid rtm_report and valid project_dir returns non-empty file dict
- [ ] TEST-026-033: persist_rtm with session_id="" skips KG snapshot write without error
- [ ] TEST-026-034: persist_rtm with kg_store=None skips KG snapshot write and still saves to disk
Coverage and Compliance Extraction
- [ ] TEST-026-035: _extract_coverage with None returns 0.0
- [ ] TEST-026-036: _extract_coverage with RTMReport lacking validation attribute returns 0.0
- [ ] TEST-026-037: _extract_coverage with valid RTMReport returns validation.coverage_percent
- [ ] TEST-026-038: _extract_compliance_pct with None returns 0.0
- [ ] TEST-026-039: _extract_compliance_pct with valid ComplianceResult returns compliance_pct
Pass/Fail Threshold Logic
- [ ] TEST-026-040: passed is True when coverage_pct > 0.0 and compliance_threshold == 0.0 (default)
- [ ] TEST-026-041: passed is True when coverage_pct == 0.0 and compliance_threshold == 0.0
- [ ] TEST-026-042: passed is False when coverage_pct < compliance_threshold set to 80.0
Report Building
- [ ] TEST-026-043: build_report_lines with no RTM and no compliance result contains "Result:" line
- [ ] TEST-026-044: build_report_lines with valid RTM includes "RTM Coverage" section with verified, missing, orphan, and coverage lines
- [ ] TEST-026-045: build_report_lines with valid ComplianceResult includes "Design Compliance" section with compliance_pct, missing_classes, and extra_classes
- [ ] TEST-026-046: build_report_lines with passed=True produces "Result: PASS" line
- [ ] TEST-026-047: build_report_lines with passed=False produces "Result: FAIL" line
Task Slug
- [ ] TEST-026-048: _task_slug converts spaces to hyphens and lowercases the result
- [ ] TEST-026-049: _task_slug replaces special characters with hyphens
- [ ] TEST-026-050: _task_slug truncates output to 50 characters maximum
- [ ] TEST-026-051: _task_slug strips leading and trailing hyphens from output
Payload Construction
- [ ] TEST-026-052: _build_payload includes passed, coverage_pct, compliance_pct, rtm_files, missing_classes, missing_reqs, project_dir, and task_summary keys
- [ ] TEST-026-053: _build_payload with None compliance_result sets missing_classes to empty list
- [ ] TEST-026-054: _build_payload with None rtm_report sets missing_reqs to empty list
Other tracked files
specs/026-compliance/README.mdspecs/026-compliance/compliance-audit.md
Full spec text
Verbatim from specs/026-compliance/spec.md; headings are pushed down two levels.
SPEC-026: Compliance Volk (U:Cp)
Metadata
- Spec ID: 026-compliance
- Title: Compliance Volk
- Author: Mohamed
- Status: Implemented
- Created: 2026-04-04
- Last Updated: 2026-04-04
- Priority: P1
- Version: 1.0.0
- Extends: (none)
Overview
What
ComplianceVolk (alias U:Cp, role "Compliance") is a deterministic volk that runs after U:Qa completes. It executes a 7-step pipeline that scans @traces decorators from source and test code, loads requirements from the Neo4j knowledge graph, generates a Requirements Traceability Matrix (RTM), syncs code nodes back to the KG, checks design-vs-code compliance, persists the RTM to disk and the KG, and produces a compliance report with coverage % and compliance % metrics.
Why
- Traceability between requirements and implementation must be verified automatically after every code generation cycle
- Design drift (classes specified in the KG but absent in code, or vice versa) must be detected before delivery
- RTM must be persisted for audit and human review without manual effort
- The compliance step must be deterministic — no LLM inference introduces non-determinism into governance decisions
- The pipeline integrates directly after U:Qa in the orchestration chain, closing the loop from intent to verification
How
- ComplianceVolk extends BaseVolk with alias
U:Cpand roleCompliance - ComplianceVolk routes CXP packets to subject
complianceto the internal pipeline - All heavy logic is extracted to
_pipeline.pyas pure functions with no shared state - KG connection is lazy-initialised via
get_kg_store()usingKGDesignStore; failure is non-fatal (kg_store set to None, pipeline degrades gracefully) - Response is a CXP packet sent to HUMAN with coverage and compliance metrics
- RTM is saved to
project/docs/as JSON and markdown, and as an RTMSnapshot node in Neo4j
Terminology
| Term | Definition |
|---|---|
| RTM | Requirements Traceability Matrix — cross-reference of requirements to implementing code |
| TraceInfo | Dataclass from capturing annotation data: spec, story, reqs, module, class_name, method, file_path, line_number |
| Requirement | Dataclass from representing a single requirement with id, description, story_id, spec_id, priority, category |
| RTMReport | Dataclass from containing entries list, ValidationReport, generated_at, and project slug |
| ValidationReport | Dataclass tracking verified, missing, and orphan requirement IDs plus coverage_percent |
| ComplianceResult | Dataclass from with missing_classes, extra_classes, and compliance_pct |
| RTMSnapshot | Dataclass from persisted as a Neo4j node linked to the session |
| KGDesignStore | Neo4j-backed design store from orchestrata.knowledge.kg_design; provides get_design_classes, sync_code_nodes, create_rtm_snapshot, get_requirements |
| Task slug | URL-safe lowercase string derived from task_summary; used as project identifier in KG lookups |
| Design compliance | Ratio of expected classes (from KG design nodes) that exist in code, expressed as a percentage |
| Coverage % | Percentage of requirements that have at least one annotation pointing to them |
| Orphan trace | A annotation referencing a requirement ID that does not exist in the loaded requirements |
| Deterministic volk | A volk that produces the same output for the same inputs; makes no LLM calls |
Package Layout
File Responsibilities
| File | Responsibility |
|---|---|
volk.py |
ComplianceVolk class, CXP packet routing, pipeline orchestration, report building, module-level helpers |
_pipeline.py |
Pure helper functions: scan_traces, load_requirements, generate_rtm, sync_code, check_compliance, persist_rtm, _calculate_results, and their private sub-functions |
__init__.py |
Module docstring declaring the package |
User Scenarios & Stories
US-001: Automated RTM Generation (P1)
As a CodeDirector pipeline I want the compliance step to generate an RTM automatically after U:Qa completes So that traceability coverage is verified without manual effort
Acceptance Criteria:
- AC-001-01: Given a CXP packet with and , when processed, then all decorators in and are scanned
- AC-001-02: Given scanned traces and KG requirements, when RTM is generated, then each requirement is classified as verified, missing, or orphan
- AC-001-03: Given a generated RTM, when persisted, then JSON and markdown files are written to
- AC-001-04: Given a generated RTM, when the pipeline completes, then coverage % appears in the response payload
- AC-001-05: Given KG is unavailable, when the pipeline runs, then it degrades gracefully and still returns a result
US-002: Design-vs-Code Compliance Check (P1)
As a governance system I want design classes specified in the KG to be verified against actual code So that design drift is detected before delivery
Acceptance Criteria:
- AC-002-01: Given design nodes in the KG and a project slug, when runs, then missing and extra classes are identified
- AC-002-02: Given compliance results, when the pipeline finishes, then compliance % appears in the response payload
- AC-002-03: Given KG unavailable, when compliance check is skipped, then compliance % is reported as 0.0 without crashing
US-003: Code Sync to Knowledge Graph (P1)
As the knowledge graph I want scanned trace records to be synced as code nodes after each compliance run So that the KG reflects the current state of the implementation
Acceptance Criteria:
- AC-003-01: Given trace records from the scan step, when sync_code runs, then all trace dicts are sent to kg_store.sync_code_nodes
- AC-003-02: Given KG unavailable, when sync is skipped, then the pipeline continues without error
US-004: RTM Persistence to KG (P1)
As an audit trail I want RTM snapshots persisted as Neo4j nodes linked to the session So that historical compliance metrics are queryable over time
Acceptance Criteria:
- AC-004-01: Given a valid RTM and a session_id, when persist_rtm runs, then an RTMSnapshot node is created in Neo4j with coverage_pct, verified_count, missing_count, and orphan_count
- AC-004-02: Given no session_id, when persist_rtm runs, then the KG write is skipped without error
- AC-004-03: Given RTM is None, when persist_rtm is called, then a zero-item dict is returned immediately
US-005: Human-Readable Compliance Report (P2)
As a developer I want a formatted compliance report in the CXP response artifact So that I can read coverage and compliance status at a glance
Acceptance Criteria:
- AC-005-01: Given a compliance run that completes, when the report is built, then it includes RTM Coverage section with verified/missing/orphan counts and coverage %
- AC-005-02: Given a compliance run that completes, when the report is built, then it includes Design Compliance section with compliance %, missing classes, and extra classes
- AC-005-03: Given a compliance run that completes, when the report is built, then it ends with a or line
Data Models
CXP Inbound Packet (ComplianceVolk.process)
| Field | Type | Required | Description |
|---|---|---|---|
| project_dir | string | yes | Absolute path to project root; src/ and tests/ sub-dirs are derived from this |
| task_summary | string | yes | Human-readable task description; converted to slug for KG lookups |
| session_id | string | no | Current session ID; used to link RTMSnapshot to Neo4j session node |
CXP Outbound Packet (ComplianceVolk.response)
| Field | Type | Description |
|---|---|---|
| passed | bool | True if coverage_pct >= compliance_threshold (default 0.0) |
| coverage_pct | float | Percentage of requirements with at least one trace annotation |
| compliance_pct | float | Percentage of design classes present in code |
| rtm_files | dict | Mapping of file type ("json", "markdown") to absolute path |
| missing_classes | list[str] | Classes found in KG design nodes but absent in code |
| missing_requirements | list[str] | Requirement IDs with no trace annotation |
| project_dir | string | Echo of input project_dir |
| task_summary | string | Echo of input task_summary |
TraceInfo (from orchestrata.core.traceability_utils)
| Field | Type | Description |
|---|---|---|
| spec_id | str | Specification ID the trace points to |
| story_id | str | User story ID |
| requirement_ids | list[str] | Requirement IDs covered by this annotation |
| module_path | str | Python module path |
| class_name | str | Name of the annotated class or blank string |
| method_name | str | Name of the annotated method or blank string |
| file_path | str | Absolute path to the source file |
| line_number | int | Line number of the annotation |
Requirement (from orchestrata.core.traceability_utils)
| Field | Type | Description |
|---|---|---|
| uid | str | Requirement identifier matching REQ-NNN-NNN pattern |
| description | str | Human-readable requirement text |
| story_id | str | Parent user story ID |
| spec_id | str | Parent specification ID |
| priority | str | Requirement priority (default "NORMAL") |
| category | str | Requirement category tag |
RTMReport (from orchestrata.core.traceability_utils)
| Field | Type | Description |
|---|---|---|
| list[dict] | RTM entry dicts with req_id, trace list, status | |
| ValidationReport | None | Aggregated coverage metrics | |
| str | ISO 8601 timestamp of generation | |
| str | Project slug |
ValidationReport (from orchestrata.core.traceability_utils)
| Field | Type | Description |
|---|---|---|
| verified_ids | list[str] | Requirement IDs with at least one trace |
| missing_ids | list[str] | Requirement IDs with no trace |
| orphan_ids | list[str] | Traced requirement IDs not in loaded requirements |
| duplicate_ids | dict[str, list[str]] | Requirement IDs traced more than once |
| coverage_percent | float (property) | Percentage of verified requirements |
ComplianceResult (from orchestrata.knowledge.kg_design)
| Field | Type | Description |
|---|---|---|
| list[str] | Class names in KG design but absent in code | |
| list[str] | Class names in code not present in KG design | |
| float | Percentage of design classes present in code |
RTMSnapshot (from orchestrata.knowledge.kg_design)
| Field | Type | Description |
|---|---|---|
| uid | str | Unique snapshot ID ("rtm-" + 8 hex chars) |
| coverage_pct | float | RTM coverage percentage at snapshot time |
| verified_count | int | Number of verified requirements |
| total_count | int | Total requirements (verified + missing) |
| missing_count | int | Number of missing requirements |
| orphan_count | int | Number of orphan trace annotations |
Functional Requirements
Pipeline Orchestration
- FR-001: ComplianceVolk.process() routes packets with subject
complianceto the internal pipeline; all other subjects return None - FR-002:
_run_pipeline()callsscan_traces,load_requirements, andgenerate_rtmin sequence - FR-003:
_run_pipeline()executes all 7 steps in order: scan_traces, load_requirements, generate_rtm, sync_code, check_compliance, persist_rtm, _calculate_results - FR-004:
_run_pipeline()transitions fromIDLEtoRUNNINGon packet receipt and toCOMPLETEDon completion - FR-005: The volk summary string is updated to reflect PASS/FAIL status and coverage/compliance percentages
Step 1 — Trace Scanning
- FR-006:
scan_tracesscans@tracesdecorators in Python files under src_dir and tests_dir usingfind_py_filesandextract_trace_info - FR-007: Scanning is skipped for any directory that does not exist (
path.exists()check); no exception is raised - FR-008: Trace records from both src_dir and tests_dir are merged into a single list
- FR-009: If the scan raises an exception, a zero-item list is returned and the error is logged at WARNING level
Step 2 — Requirements Loading
- FR-010:
load_requirementsfetches requirements from the KG viakg_store.get_requirements - FR-011: If kg_store is None, load_requirements returns a zero-item list immediately without attempting a KG call
- FR-012: KG requirement dicts are converted to
Requirementdataclass instances with id, description, story_id, spec_id fields - FR-013: If the KG call raises an exception, a zero-item list is returned and the error is logged at WARNING level
Step 3 — RTM Generation
- FR-014:
generate_rtmcallsbuild_rtm_reportwith the trace list, requirement list, and project slug - FR-015: If generation raises an exception, None is returned and the error is logged at WARNING level
- FR-016: On success, RTM validation metrics (verified, missing, orphan, coverage %) are logged at INFO level
Step 4 — Code Sync
- FR-017:
sync_codeconverts each TraceInfo to a dict viatrace_to_dictif available, then callskg_store.sync_code_nodes - FR-018: If kg_store is None, sync_code returns immediately without action
- FR-019: If the KG call raises an exception, the error is logged at WARNING level and execution continues
Step 5 — Compliance Check
- FR-020:
check_compliancecallscompute_complianceand returns the ComplianceResult - FR-021: If kg_store is None, check_compliance returns None immediately
- FR-022: On success, compliance metrics (compliance %, missing count, extra count) are logged at INFO level
- FR-023: If the KG call raises an exception, None is returned and the error is logged at WARNING level
Step 6 — RTM Persistence
- FR-024:
persist_rtmcalls_persist_rtm_to_diskand_persist_rtm_to_kgin sequence - FR-025: If rtm_report is None, persist_rtm returns a zero-item dict immediately
- FR-026:
_persist_rtm_to_diskcallswrite_rtm_jsonandwrite_rtm_markdowntargetingproject/docs/rtm.jsonandproject/docs/rtm.md - FR-027:
_persist_rtm_to_kgcreates an RTMSnapshot with a unique ID (generate_snapshot_id()) and callskg_store.create_rtm_snapshot - FR-028: If session_id is blank or kg_store is None, the KG snapshot write is skipped without error
Step 7 — Report Production
- FR-029:
_calculate_resultsextracts coverage_pct fromvalidation_reportand compliance_pct fromcompliance_result; both default to 0.0 if unavailable - FR-030:
passedis True when coverage_pct >= compliance_threshold (default 0.0) - FR-031:
build_reportbuilds a human-readable list of strings with RTM Coverage, Design Compliance, RTM Files, and Result sections - FR-032: The completed report is appended to
response.artifacts["compliance_report"]
KG Lazy Initialisation
- FR-033:
get_kg_storeimports and constructsKGDesignStoreonly on first call; subsequent calls are no-ops if kg_store is already set - FR-034: If KGDesignStore construction raises any exception, kg_store is set to None and the error is logged at WARNING level
Task Slug
- FR-035:
to_task_slugconverts task_summary to lowercase, replaces all non-word/non-hyphen characters with hyphens, strips leading/trailing hyphens, and truncates to 50 characters
Error Conditions and Edge Cases
| ID | Condition | Behaviour |
|---|---|---|
| ERR-001 | KG unavailable at init | kg_store is set to None; steps 2, 4, 5, 6 skip KG calls |
| ERR-002 | src_dir or tests_dir directory absent | Directory scan skipped; zero-item list merged |
| ERR-003 | KG returns zero-item requirements | RTM generated with all traces classified as orphans |
| ERR-004 | RTM generation raises exception | persist_step returns zero-item dict; coverage_pct = 0.0 |
| ERR-005 | Compliance check raises exception | compliance_result is None; compliance_pct = 0.0 |
| ERR-006 | project_dir blank string | Derived src_dir and tests_dir are relative paths; scans find nothing |
| ERR-007 | session_id blank string | KG RTM snapshot write skipped |
| ERR-008 | compliance_threshold = 0.0 (default) | passed = True regardless of coverage |
| ERR-009 | RTM has no validation | RTM Coverage section omitted from report |
| ERR-010 | ComplianceResult is None | Design Compliance section omitted from report |
| ERR-011 | Trace scan raises exception | Zero-item trace list returned; error logged at WARNING |
| ERR-012 | KG requirements fetch raises exception | Zero-item requirements list returned; error logged at WARNING |
| ERR-013 | Code sync to KG raises exception | Sync skipped; error logged at WARNING; pipeline continues |
API Contracts
ComplianceVolk.process()
def process(self, packet: CXPPacket) -> CXPPacket | None:
"""
Process a CXP compliance packet.
Args:
packet: CXPPacket with subject='compliance', payload containing
project_dir (str), task_summary (str), session_id (str, optional)
Returns:
CXPPacket addressed to HUMAN with compliance results, or None if
packet subject is not 'compliance'
"""
scan_traces()
def scan_traces(src_dir: str, tests_dir: str) -> list[TraceInfo]:
"""
Scan for @traces decorators in source and test files.
Args:
src_dir: Absolute path to source directory
tests_dir: Absolute path to tests directory
Returns:
List of TraceInfo dataclass instances
"""
load_requirements()
def load_requirements(kg_store: KGDesignStore | None) -> list[Requirement]:
"""
Load requirements from Neo4j knowledge graph.
Args:
kg_store: KGDesignStore instance or None
Returns:
List of Requirement dataclass instances
"""
generate_rtm()
def generate_rtm(
traces: list[TraceInfo],
requirements: list[Requirement],
project_slug: str
) -> RTMReport | None:
"""
Generate Requirements Traceability Matrix.
Args:
traces: List of trace annotations from code
requirements: List of requirements from KG
project_slug: URL-safe project identifier
Returns:
RTMReport with traceability matrix, or None on failure
"""
Non-Functional Requirements
Performance
- NFR-001: Trace scanning of 500 Python files must complete in under 10 seconds
- NFR-002: KG requirement load must complete in under 2 seconds for up to 2000 requirements
- NFR-003: Full 7-step pipeline must complete in under 30 seconds for a standard project (500 source files, 1000 test functions)
Reliability
- NFR-004: No step in the pipeline raises an uncaught exception to the caller; all failures are caught, logged at WARNING, and produce safe fallback values
- NFR-005: KG unavailability must not prevent RTM generation or report production
Determinism
- NFR-006: ComplianceVolk makes zero LLM API calls; all outputs are determined by inputs and KG state alone
- NFR-007: Given identical inputs and KG state, the compliance report is byte-for-byte reproducible
Technical Design
Architecture
ComplianceVolk extends BaseVolk with the following design:
- Stateless pipeline execution via pure functions in _pipeline.py
- Lazy KG connection via get_kg_store() singleton pattern
- Graceful degradation when KG is unavailable
- Deterministic output based solely on inputs and KG state
- 7-step sequential pipeline: scan → load → generate → sync → check → persist → report
Key Interfaces
ComplianceVolk Public API
class ComplianceVolk(BaseVolk):
alias: str = "U:Cp"
role: str = "Compliance"
def __init__(self, compliance_threshold: float = 0.0):
"""Initialize ComplianceVolk with optional compliance threshold."""
def process(self, packet: CXPPacket) -> CXPPacket | None:
"""Route compliance packets through the 7-step pipeline."""
@property
def summary(self) -> str:
"""Return compliance status summary."""
Pipeline Helper Functions (_pipeline.py)
def scan_traces(src_dir: str, tests_dir: str) -> list[TraceInfo]:
"""Scan @traces decorators in source and test files."""
def load_requirements(kg_store: KGDesignStore | None) -> list[Requirement]:
"""Load requirements from Neo4j knowledge graph."""
def generate_rtm(traces: list[TraceInfo],
requirements: list[Requirement],
project_slug: str) -> RTMReport | None:
"""Generate Requirements Traceability Matrix."""
def sync_code(traces: list[TraceInfo],
kg_store: KGDesignStore | None) -> None:
"""Sync trace records as code nodes to KG."""
def check_compliance(project_slug: str,
kg_store: KGDesignStore | None) -> ComplianceResult | None:
"""Check design-vs-code compliance via KG."""
def persist_rtm(rtm_report: RTMReport | None,
session_id: str,
kg_store: KGDesignStore | None) -> dict[str, str]:
"""Persist RTM to disk and KG."""
def _calculate_results(validation_report: ValidationReport | None,
compliance_result: ComplianceResult | None) -> dict[str, float]:
"""Calculate coverage and compliance percentages."""
Integration Points
| Integration | Direction | Interface |
|---|---|---|
| U:Qa (QA Volk) | Upstream trigger | U:Qa sends compliance packet on task completion |
| orchestrata.core.traceability_utils | Import | TraceInfo, Requirement, RTMReport, ValidationReport, build_rtm_report, find_py_files, extract_trace_info |
| orchestrata.knowledge.kg_design.KGDesignStore | Import (lazy) | get_design_classes, sync_code_nodes, create_rtm_snapshot, get_requirements |
| Neo4j (via KGDesignStore) | Write | RTMSnapshot node, code sync nodes |
| Disk (project docs/) | Write | rtm.json, rtm.md |
CXP Integration
ComplianceVolk participates in the CXP protocol as follows:
- Inbound: Receives CXPPacket with subject="compliance", containing project_dir, task_summary, and optional session_id
- Processing: Routes to internal 7-step pipeline
- Outbound: Sends CXPPacket to recipient="HUMAN" with compliance_report in artifacts, passed/coverage_pct/compliance_pct in payload
The response packet is addressed to HUMAN, not back to U:Qa, because compliance results are a human-visible deliverable. U:Ox may intercept the packet for audit purposes.
Examples
Example 1: Basic Compliance Check
Input CXP Packet:
{
"recipient": "U:Cp",
"subject": "compliance",
"payload": {
"project_dir": "/home/user/project",
"task_summary": "Implement user authentication",
"session_id": "session-abc123"
}
}
Output Response:
{
"recipient": "HUMAN",
"subject": "compliance",
"payload": {
"passed": true,
"coverage_pct": 85.5,
"compliance_pct": 92.3,
"rtm_files": {
"json": "/home/user/project/docs/rtm.json",
"markdown": "/home/user/project/docs/rtm.md"
},
"missing_classes": ["UserManager"],
"missing_requirements": ["REQ-026-003", "REQ-026-007"]
}
}
Example 2: Graceful Degradation (KG Unavailable)
Scenario: Neo4j is down
Result: Pipeline completes with degraded metrics:
- coverage_pct: 0.0 (no requirements loaded)
- compliance_pct: 0.0 (no design comparison)
- RTM files still generated with orphan classifications
- No KG writes attempted
Example 3: Zero-File Project
Input: project_dir with no Python files
Output:
- trace list has no items
- RTM report has zero entries
- coverage_pct: 0.0 (no verified requirements)
- compliance_pct: 0.0 (no classes to compare)
Security
Authentication & Authorization
- ComplianceVolk operates within the orchestrata agent framework with implicit trust
- All CXP packet routing is mediated by the orchestrata runtime
- No direct user authentication required at the volk level
Input Validation
- All paths validated via
pathlib.Path.exists()before filesystem access - KG connection failures are handled gracefully (never raise uncaught exceptions)
- Task summaries sanitized via
to_task_slug()to prevent injection attacks - File writes restricted to project_dir/docs/ subdirectory only
Audit Logging
- All KG operations logged at INFO level with operation details
- Exception conditions logged at WARNING level with error context
- Compliance results logged with coverage/compliance metrics
- No sensitive data (credentials, keys) logged
- RTM snapshots linked to session_id for audit trail
Data Privacy
- RTM files written to project-local docs/ directory only
- KG sync operations only write trace metadata (no code content)
- Session linkage via session_id enables privacy-aware audit trails
- No user data processed; only technical compliance metrics
Governance
Compliance Monitoring
- Coverage Threshold: Default 0.0 (configurable via compliance_threshold)
- Success Criteria: coverage_pct >= compliance_threshold
- Metrics Tracked: Coverage %, Compliance %, missing/extra classes, orphan traces
- Historical Tracking: RTM snapshots persisted to KG for trend analysis
Quality Gates
- All exceptions caught and converted to safe defaults (no cascading failures)
- KG unavailability never blocks RTM generation or report production
- Zero-item project directories handled gracefully (no errors, just zero metrics)
- File write failures logged but don't crash the pipeline
Operational Controls
- Lazy KG initialization prevents startup failures
- Pipeline steps designed as pure functions for testability
- All public functions have Google-style docstrings for documentation
- Module structure enables targeted testing without Neo4j dependency
Change Management
- Spec versioning follows semantic versioning (MAJOR.MINOR.PATCH)
- Functional requirements are immutable once implemented
- Breaking changes require new spec version with migration notes
- All changes require update to Last Updated metadata field
Success Criteria
Pipeline Execution
- [ ] ComplianceVolk processes CXP packets with subject "compliance" and routes to internal pipeline
- [ ] All 7 pipeline steps execute in sequence: scan_traces → load_requirements → generate_rtm → sync_code → check_compliance → persist_rtm → _calculate_results
- [ ] Pipeline completes within 30 seconds for standard projects (500 source files, 1000 test functions)
- [ ] No uncaught exceptions propagate to caller; all failures produce safe fallback values
RTM Generation
- [ ] All @traces decorators in src/ and tests/ directories are scanned and extracted
- [ ] Requirements loaded from Neo4j KG are matched against trace annotations
- [ ] Each requirement classified as verified, missing, or orphan
- [ ] RTM persisted to disk as JSON (project/docs/rtm.json) and markdown (project/docs/rtm.md)
- [ ] Coverage % calculated and included in response payload
Design Compliance
- [ ] Design classes from KG compared against code classes
- [ ] Missing classes (in KG but not code) identified and reported
- [ ] Extra classes (in code but not KG) identified and reported
- [ ] Compliance % calculated and included in response payload
KG Integration
- [ ] Trace records synced to KG as code nodes after scan
- [ ] RTMSnapshot node created in Neo4j with coverage metrics
- [ ] RTMSnapshot linked to session node via session_id
- [ ] KG unavailability handled gracefully with degraded metrics (0.0 for coverage and compliance)
Report Production
- [ ] Human-readable compliance report generated with RTM Coverage section
- [ ] Report includes Design Compliance section with percentages and class lists
- [ ] Report ends with PASS or FAIL result line
- [ ] Report appended to response artifacts under "compliance_report"
Determinism
- [ ] Zero LLM API calls made during compliance processing
- [ ] Identical inputs produce byte-for-byte reproducible outputs
- [ ] Compliance decisions based solely on inputs and KG state
Error Handling
- [ ] Missing src_dir or tests_dir handled by skipping scan
- [ ] Zero-item requirements from KG handled by classifying all traces as orphans
- [ ] RTM generation failures return zero-item dict with coverage_pct = 0.0
- [ ] Compliance check failures return None with compliance_pct = 0.0
Testing
Test Infrastructure
- Tests use real ComplianceVolk instances and real dataclass instances from orchestrata.core and orchestrata.knowledge
- No mocking; pipeline functions accept None for kg_store, enabling testing without Neo4j
- FakeTraceInfo, FakeRequirement, FakeRTMReport dataclasses in test files provide deterministic test fixtures without mocking
Key Test Scenarios
- Volk instantiation: alias, role, default threshold, initial status, zero-item artifacts
- CXP routing: compliance subject returns packet; unknown subject returns None
- Pipeline with zero-item project_dir: completes without crash, returns 0.0 coverage
- to_task_slug: special characters, truncation, lowercase
- build_report / build_rtm_markdown: None inputs, missing attributes
- _calculate_results: with and without RTM, with and without compliance result
- persist_rtm: correct field mapping from report dict
- Threshold logic: passed = True when coverage >= threshold
Definition of Done
- [x] ComplianceVolk — ComplianceVolk with 7-step pipeline, CXP routing, report building
- [x] _pipeline.py — all 7 helper functions with docstrings and error handling
- [x] init.py — module docstring
- [x] All public functions have Google-style docstrings with Args/Returns
- [x] All pipeline steps catch exceptions and return safe defaults
Visual Architecture
Class Diagram
Diagram source shown as text: Mermaid could not parse it (Error: Parse error on line 15:).
classDiagram-direction LR
class ArchitectAgent {
$agent_type: ClassVar[str] = 'U:Ar'
---
+__init__(model: str = 'claude-3-5-haiku', agent_name: str = '')
+async analyze_impact(file_path: str, old_string: str, new_string: str, context: dict[str, Any] | None = None): dict[str, Any]
+async design_solution(requirements: dict[str, Any]): dict[str, Any]
+async review_architecture(architecture: dict[str, Any]): dict[str, Any]
}
class BaseAgent <<pydantic>> {
-model: str = Field(default='claude-3-5-haiku', description='Model identifier for the agent', min_length=1, strict=True)
-agent_id: str = Field(default='', description='Unique agent identifier', strict=True)
-agent_name: str = Field(default='', description='Human-readable agent name', strict=True)
$agent_type: ClassVar[str] = 'U:Base'
$_instance_counters: ClassVar[dict[str, int]] = {}
---
{@classmethod} +validate_model(v: str): str
+__init__(**data: Any): None
#_generate_unique_id(): str
+model_post_init(__context: Any): None
+async receive_message(message: Any, sender: str | None = None): Any
}
class CodeDirectorAgent {
$agent_type: ClassVar[str] = 'U:Cd'
---
+__init__(model: str = 'claude-3-5-sonnet', agent_name: str = '')
+async execute_edit(file_path: str, old_string: str, new_string: str, context: dict[str, Any] | None = None): dict[str, Any]
+async review_code(code: str, requirements: dict[str, Any]): dict[str, Any]
+async implement_feature(spec: dict[str, Any]): dict[str, Any]
}
class QualityAssuranceAgent {
$agent_type: ClassVar[str] = 'U:Qa'
---
+__init__(model: str = 'claude-3-5-sonnet', agent_name: str = '')
+async review_edit(file_path: str, old_string: str, new_string: str, context: dict[str, Any] | None = None): dict[str, Any]
+async run_tests(test_suite: str): dict[str, Any]
+async audit_security(code: str): dict[str, Any]
}
class UserExperienceAgent {
$agent_type: ClassVar[str] = 'U:Ux'
---
+__init__(model: str = 'claude-3-5-haiku', agent_name: str = '')
+async review_ux(file_path: str, old_string: str, new_string: str, context: dict[str, Any] | None = None): dict[str, Any]
+async design_ui(requirements: dict[str, Any]): dict[str, Any]
+async evaluate_accessibility(ui: dict[str, Any]): dict[str, Any]
}
BaseAgent <|-- ArchitectAgent
BaseModel <|-- BaseAgent
BaseAgent <|-- CodeDirectorAgent
BaseAgent <|-- QualityAssuranceAgent
BaseAgent <|-- UserExperienceAgent
Dependencies
| Dependency | Version | Purpose |
|---|---|---|
| orchestrata-core | workspace | TraceInfo, Requirement, RTMReport, ValidationReport, build_rtm_report |
| orchestrata-agents | workspace | BaseVolk, CXPPacket, CXP_PROTOCOL_VERSION |
| orchestrata-knowledge | workspace | KGDesignStore, ComplianceResult, RTMSnapshot |
| Python | >=3.14 | Free-threaded runtime |
Observability
Logging Requirements
- Component Logging
- Log all major operations with appropriate log levels
- Include correlation IDs for request tracing
- Log performance metrics for critical operations
- Log errors with full context for debugging
Metrics Requirements
- Performance Metrics
- Operation duration histograms
- Request/error rate counters
- Resource utilization gauges
- Business metrics relevant to domain
Tracing Requirements
- Distributed Tracing
- Trace propagation across component boundaries
- Span attributes for domain-specific context
- Trace sampling based on operational needs
Alerting Requirements
- Critical Alerts
- Error rate spikes
- Performance degradation
- Resource exhaustion
- Business metric anomalies
Dashboard Requirements
- Operational Dashboard
- Real-time system health
- Performance trends
- Error analysis
- Resource monitoring
Revision History
| Version | Date | Author | Changes |
|---|---|---|---|
| 1.0.0 | 2026-04-04 | Mohamed | Initial spec — documents implemented package |