Orchestrata Ultimate › Plans › spec-020-drift-ledger
SPEC-020 drift ledger
Source: plans/spec-020-drift-ledger.md at 8a2cb42ad · last changed 2026-08-23 · first committed 2026-08-23 · 255 lines.
Prose pending.
Plan text
SPEC-020 drift ledger
Compiled 2026-08-23. Method: SPEC-020 read in full (spec.md 1911 lines, compliance-audit.md, 4 checklists), cross-referenced against every hook declaration source in the repo and on this machine, with git history used to date each disconnection.
Finding in one line: SPEC-020 is marked Implemented, P0. Of its 10 user scenarios, 4 are entirely disconnected, 3 are partial, 3 are live. Six capabilities were registered and lost on a known date; eight were specified and never wired at all. Nothing reported it, because the reporting layer is one of the disconnected capabilities.
1. Why this went unseen
Three independent detectors that should have caught it, and why each did not:
| detector | state |
|---|---|
RTM / @traces |
Blind to claude/. 231 of 240 REQ-020 requirements grade UNLINKED, cited_by: [] — while the hooks cite those exact IDs in source (block_except_pass.py:36-44 cites REQ-020-156..165). The harvester never scans the directory the spec is implemented in, so a disconnected spec is indistinguishable from an unbuilt one. |
specs/020-claude-code-harness/compliance-audit.md |
Rotted. Reports Priority: Unknown, Status: Unknown, Implementation Source: Missing, and a class list (CircuitBreaker, ThreadPoolManager, UltimateOrchestrator) belonging to a different spec. It tracks none of SPEC-020's hooks. |
stop_auditor.py (US-003) |
Is itself one of the disconnected capabilities. The hook whose job is to report "claims not backed by tool usage" cannot report that it is off. |
The drift is self-concealing by construction. Reachability analysis reads the result as dead code — which is how this ledger's predecessor plan came to propose deleting the enforcement layer rather than restoring it.
2. The disconnection event
a7433969 — 2026-08-14 — "installer: the plugin is the only hook delivery path"
The cutover moved hook delivery from claude/settings.json to
claude/settings.hooks.json + the plugin. claude/settings.json is not dead
config: its 12 commands are exactly the registration block specified at
spec.md:551-581. It is the SPEC-020 registration manifest, and the cutover
orphaned it.
Carried over: 5 of 11. Dropped: 6.
CARRIED intercept_edit.py intercept_bash.py intercept_write.py
infra_check.py session_close_hook.py
DROPPED enforcement_hook.py log_operation.py (x2) posttool_auditor.py
session_auto_start.py session_context_hook.py stop_auditor.py
Registered since 2026-07-27; unregistered since 2026-08-14. 9 days live, then off.
The same commit's sibling note (settings.hooks.json:2, _why_intercept_bash)
records that intercept_bash/write/edit and block_file_tools.py "shipped
in the tree, deployed, and were registered NOWHERE — so they never ran." Three
were fixed that day. block_file_tools.py was named in the same sentence and
is still unregistered.
3. The ledger
A — LOST AT CUTOVER (registered 2026-07-27, dropped 2026-08-14)
| hook | spec'd event | scenario | capability lost |
|---|---|---|---|
enforcement_hook.py |
PreToolUse .* |
US-002/US-004 | central rule enforcement across all tool uses |
log_operation.py |
PostToolUse Edit and Write|Edit|Bash|Read|Glob|Grep |
US-008 | per-operation audit log |
posttool_auditor.py |
PostToolUse .* |
US-008 | tool-call accumulation into session JSONL |
session_auto_start.py |
SessionStart | US-001 | session auto-creation, compaction-duplicate guard |
session_context_hook.py |
SessionStart | US-001 | session context injection (drives context_graph/) |
stop_auditor.py |
Stop | US-003 | deception detection: claims vs actual tool usage |
B — SPEC'D, NEVER WIRED (since spec authored 2026-03-29)
| hook | spec'd event | scenario | capability never delivered |
|---|---|---|---|
error_gate_hook.py + error_gate/{checks_syntax,checks_lint,checks_test,checks_structure,utils} |
Stop | US-002 (P0) | 6 check categories; blocks session end on errors |
read_throttle.py |
PreToolUse Read|Grep |
US-007 (P2) | 3 reads / 30s, redirect to KG |
block_except_pass.py |
PreToolUse Write|Edit |
US-004 (P0) | AST-based bare-except / except: pass block |
block_file_tools.py |
PreToolUse Read|Grep |
US-005 (P1) | Neo4j context injection on read |
sync_context_md.py |
PostToolUse (on context.json) |
US-006 (P0) | context.md auto-regeneration |
deception_detector.py |
(support module) | US-003 | deception pattern library |
instruction_enforcer_hook.py |
PreToolUse | US-002/US-004 | instruction enforcement |
stop_auditor_subagent.py |
SubagentStop | US-003 | separate subagent audit |
C — LIVE
intercept_edit/bash/write (repaired 2026-08-14), infra_check,
session_close_hook, session_open_hook, remember_hook, plus 20 further
hooks registered in settings.hooks.json that post-date the spec.
4. Capability status by scenario
| scenario | prio | status | evidence |
|---|---|---|---|
| US-001 Session lifecycle | P0 | PARTIAL | close/open/resume live; session_auto_start + session_context_hook lost at cutover |
| US-002 Quality enforcement | P0 | DEAD | error_gate_hook never wired; enforcement_hook lost |
| US-003 Deception detection | P1 | DEAD | stop_auditor lost; detector + subagent never wired; ~/.claude/audit/ does not exist |
| US-004 Safety guards | P0 | PARTIAL | intercepts live; block_except_pass never wired; audit logging lost |
| US-005 Neo4j context injection | P1 | DEAD | block_file_tools never wired |
| US-006 Context persistence | P0 | PARTIAL | context.json live via CLI; sync_context_md never wired |
| US-007 Read throttling | P2 | DEAD | read_throttle never wired |
| US-008 Audit trail | P1 | DEAD | both loggers lost; the spec's "NEVER deleted" archive tree was never created |
| US-009 RTM generation | P1 | LIVE (behaviour unverified) | session_close_hook registered |
| US-010 Remember/search | P2 | LIVE (behaviour unverified) | remember_hook registered |
~/.claude/audit/ does not exist. The spec mandates
audit/{sessions,archive,findings} with "Archive data NEVER deleted". No
deception_log.jsonl has ever been written.
5. What this changes
The predecessor plan (plans/claude-surface-pipeline.md) classified these 40
modules by reachability, labelled them "264 KB of dead weight", and proposed
deletion. That plan is void. The correct reading of the standing ruling is its
second clause — delete it or WIRE it — and for eight unwired P0/P1
scenarios in a spec marked Implemented, wiring is the only defensible branch.
The work is a spec-conformance restoration, in three parts:
- Restore — re-register category A (the six lost at cutover; they ran for 9 days, so the wire-vs-retire question is already answered by history) and decide category B per hook against the spec.
- Make it visible — the RTM harvester must scan
claude/; the rotted compliance audit must work or be deleted. - Make it fail loudly — a gate that fails when a spec marked Implemented has requirements whose implementing hook is registered nowhere. That gate is what was missing, and it is what stops recurrence.
6. The widened check — other specs
Run 2026-08-23. Three distinct failure modes separate cleanly; only the third is the SPEC-020 defect.
6.1 RTM blindness — bounded, and SPEC-020 is the only material victim
generate_rtm_evidence.py defaults: --src src/orchestrata_ultimate,
--tests tests, --web opencode,vizint-ui. Never scanned: claude/,
scripts/, src/orchestrata_ast/, src/orchestrata_tools/, deploy/,
prolog/. That hides 607 citations.
| spec | cited in scanned roots | cited in blind roots |
|---|---|---|
| 020 | 42 | 365 |
| 021 | 315 | 61 |
| 025 | — | 25 |
| 033 | — | 22 |
| 109 / 001 | — | 16 each |
Only SPEC-020 is majority-invisible (90%). For every other spec the blind citations are a minority. Negative finding: the blindness does not hide a second disconnected spec.
6.2 Never traced — large, but a different defect
The high-UNLINKED specs are not blind and not renumbered; their requirements were simply never cited. The ID-mismatch hypothesis is refuted (SPEC-007: 107 of 112 cited IDs match known IDs; SPEC-016: 38/38; SPEC-014: 100/100).
| spec | status | requirements | cited | uncited |
|---|---|---|---|---|
| 016 VizInt Command Center | Implemented | 294 | 38 | 256 |
| 007 Prolog Governance | Complete | 368 | 112 | 256 |
| 101 Ultimate Hybrid Arch | Complete | 245 | 1 | 244 |
| 099 Security & Auth | Complete | 171 | 1 | 170 |
| 062 / 053 / 098 | Complete | 125 / 98 / 96 | 0 | all |
This is a traceability-coverage gap, not a capability loss. Out of scope for this ledger; recorded so it is not rediscovered as drift.
6.3 The gate battery — IS invoked; the defect is coverage, not wiring
Correction. An earlier pass of this ledger claimed scripts/gates.py was
"invoked by nothing" and listed 18 gates as never run. That was wrong. It was
derived from searching deploy/, scripts/, .github/,
.pre-commit-config.yaml and pyproject.toml — and never looking in
.claude/hooks/, where the invocation contract lives.
How the battery is actually driven:
.claude/hooks/single-gate-entrypoint.sh— a live PreToolUse(Bash) guard, registered at.claude/settings.json:51. It DENIES running gate scripts individually and redirects topython scripts/gates.py, because "invoking the gate scripts individually is how a ~25s parallel battery turns into a ~50s serial one, and how gates get silently dropped from a 'full' review."- The battery is therefore agent-invoked on demand, not a CI step.
.cache/gates.jsonrecords a fingerprinted verdict per gate. Last written 2026-08-20 11:25: 24 gates cached, 22ok:true, 2ok:false(mypy,prolog-gate).
So the battery runs, records failures, and skips gates whose inputs are unchanged. The two cached FAILs are unresolved as of this ledger.
The real defects are two false-greens, both confirmed by running the gate:
check_prod_tree_separation.py(RUL-027) runs and reportsOK — RUL-027 satisfied: 7 unit(s), none reference …/workspace/…— while~/.config/systemd/user/orchgraph-backup.servicecarriesEnvironment=ORCHGRAPH_BACKUP_DIR=/home/mohamed/workspace/orchestrata-ultimate-architecture/…and writes into the dev checkout daily at 03:00. Cause:DEFAULT_GLOB = "orchestrata-*.service"(line 50) excludesorchgraph-*, whileship.sh:592discovers^(orchestrata|orchgraph)-. A live violation passing a gate that ran.check_feature_loss.pyruns, and did not catch SPEC-020. Its subject is deletion — "a REQ-* must never lose its implementation silently", motivated by commit6a186c9ddeleting 9 files / 2,392 lines. SPEC-020's hooks were never deleted; they were un-registered, files intact. The gate detects loss-by-deletion and is blind to loss-by-disconnection.
Its docstring quotes the user ruling of 2026-08-19 verbatim: "add checks at the gates so we never face the lost code and features again and make them non negotiable, make them all reference the specs and report loudly any deleted features and loss." The gate honours the letter (deleted features) and misses the case that actually occurred. It also names the precise error this ledger's predecessor plan repeated: an audit that "mistook 'no importer' for 'no consumer'."
6.4 The pattern
One enforcement layer is disconnected (the SPEC-020 harness, §2). The other runs but cannot see the failure mode that disconnected it:
| layer | state | why it missed SPEC-020 |
|---|---|---|
| SPEC-020 hooks | 6 unregistered at cutover, 8 never wired | it is the loss |
| gate battery | runs; 22 PASS / 2 FAIL @ 2026-08-20 | check_feature_loss detects deletion, not disconnection |
| RTM | runs | blind to claude/ — 365 citations invisible |
| compliance audit | rotted | tracks none of SPEC-020's hooks |
The missing check is one sentence long and no existing gate expresses it: a requirement whose implementing hook is present in the tree but registered in no declaration source is a loss. Deletion is covered. Disconnection is not.
7. Still not established
- Whether any category-B hook or orphaned gate was deliberately shelved. No
rationale found in the commits examined;
history_intel.py whyreturnsno-recorded-rationaleas a finding, not a failure. - Behaviour verification for US-009 and US-010 — registration confirmed, output not checked.
- SPEC-006 and SPEC-008, which SPEC-020 extends — not yet examined.
- Whether the 18 orphaned gates would PASS if run. Several are known to fail
now (
check_prod_tree_separation,gen_plugin --checkdrift, and the_normalize_depstdlib-edge regression in the working tree), so wiring them is a two-step: wire, then fix what they report.