Orchestrata Ultimate › Plans › spec-audit › _METHOD

Spec-audit method (shared brief for all investigation agents)

Source: plans/spec-audit/_METHOD.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-04 · 63 lines.

Prose pending.

Plan text

Spec-audit method (shared brief for all investigation agents)

You are one of 11 parallel investigation agents in a planning session. Your job is to investigate & verify an assigned cluster of specs against the actual implementation and produce a per-spec audit + implementation plan. This is READ-ONLY investigation of src/: you may run tools and read code, but you must NOT modify any source/test file. Your only write is your cluster report file.

Golden rules

1. Evidence over markers. The specs/NNN-*/checklists/requirements.md - [ ] boxes are STALE and unreliable — ignore them. Judge by code only. Every claim needs file:line evidence.
2. Use the AST toolkit, not grep/sed, for structure questions. From repo root:
   • python scripts/serialize_codebase_ast.py symbols --pattern <Name> — does a symbol exist?
   • python scripts/serialize_codebase_ast.py refs <Name> — is it actually wired/called? (0 refs = orphaned)
   • Also: check-deps, standards, unused. Read bodies directly only for logic the AST can't capture.
3. The port is essentially complete. Most specs are largely DONE. Your value is finding the real remaining work — not re-listing what already works. Be honest; no false confidence.
4. Do not implement anything. No code edits, no fakes/mocks/stubs. Planning only.
5. Build on prior work. docs/spec-gap-register.md already audited 20 core specs (002,003,004,008,009,010,011,012,013,014,015,016,023,026,027,028,029,030,031,032). If one of yours is there, CONFIRM/UPDATE against current code (cite the register), don't blindly redo.

Classification (pick one per spec)

• DONE — all material capabilities implemented AND wired (refs > 0). No action.
• PARTIAL — core works but named capabilities missing/unwired. Has remaining work.
• GAP — a required capability is genuinely absent/broken.
• DIVERGENCE — code intentionally differs from spec; spec is out of date (cite the code comment/design that proves intent). Action = update the spec, not the code.
• ORPHANED — implemented but 0 refs / duplicate subsystem; needs consolidation or deletion.

Per-spec output format (in your report file)

For EACH spec in your cluster, write a section:

### SPEC-NNN <name> — <CLASSIFICATION>
- **Specifies:** one line.
- **Verified:** the major required capabilities, each with ✅/⚠️/❌ + `file:line` evidence
  (name the symbol you checked via `symbols`/`refs`).
- **Remaining work / plan:** (only if not DONE)
  - Scope: what to build/fix, concretely.
  - Target files: `src/...` paths to create/modify.
  - Approach: 2–4 sentences.
  - Dependencies: which OTHER specs/subsystems this couples to (critical for parallelism analysis).
  - Effort: S (<0.5d) / M (0.5–2d) / L (>2d). Risk: low/med/high + why.
- **Evidence log:** the exact AST/read commands you ran and what they showed.

End your report with a cluster summary table: | Spec | Class | Effort | Hard deps |.

What to return to the orchestrator (your final message)

A compact digest (NOT the whole file): one line per spec — SPEC-NNN: CLASS, effort, key deps. Plus: which specs in your cluster have real remaining work, and any cross-cluster dependencies you noticed (e.g. "SPEC-069 task tools depend on SPEC-023 goal-system controller choice").

Project constraints (context you may need)

• Python 3.14 free-threaded; uv only (pip blocked); absolute imports (orchestrata_ultimate.*).
• Source of truth for target code: src/orchestrata_ultimate/.
• No pytest-xdist. Real infra (Redis .10 / Neo4j .11 via Incus), no mocks.
• Governance runs on Prolog via janus (not pyswip).

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks