Orchestrata Ultimate › Plans › briefs › qwen › Q2a-report

Q2a Report — the AST snapshot records literal program names at process-spawn sites

Source: plans/briefs/qwen/Q2a-report.md at 8a2cb42ad · last changed 2026-10-11 · first committed 2026-10-11 · 153 lines.

Prose pending.

Plan text

Q2a Report — the AST snapshot records literal program names at process-spawn sites

1. Files changed (one-line purpose each)

No other file changed by me. git diff --cached --stat is exactly the five paths above.

2. Step-0 baseline confirmation (§7 step 0)

git status --porcelain before step 1 — exactly the expected state:

 M plans/ast-tests.json
 M plans/ast-ultimate.json
?? plans/briefs/qwen/

Orchestrator-measured baseline (from the spec, not redone): regen 1 = full reparse (93 s), regen 2 = warm (10 s), plans/ast-tests.json = 2078 modules, scripts/gate.py run = PASS, 5 rule clause(s), no violations, 426,176 facts across 23 predicates, rc 0; sizes 89,569,413 / 60,514,609 bytes.

3. Build (§7)

First regeneration (§7.4) — full reparse, as required; this is the end-to-end proof the §4.4 captures check fires (the base snapshot was warm against this worktree before the change):

Serialized orchestrata-ultimate:
  Modules: 3328
  Classes: 6452
  Functions: 29912
  Methods: 25260
  LOC: 1114175
  Output: /home/mohamed/workspace/wt-ARGV-GATE/plans/ast-ultimate.json
  Incremental: fallback (full reparse) — reused 0, reparsed 3328, removed 0
▶ Derived .../plans/ast-tests.json from ast-ultimate.json (2079 modules)
WALL=128s

Second regeneration (§7.5), no edits in between — warm, identity stable between runs:

  Incremental: warm — reused 3328, reparsed 0, removed 0
▶ Derived .../plans/ast-tests.json from ast-ultimate.json (2079 modules)
WALL=13s

3328 = the 3327 base modules + the new test module, and the tests projection grew 2078 → 2079, exactly the §7.5 expectation.

§7.6 measurements (verbatim output)

plans/ast-ultimate.json captures= ['env_accesses', 'argv_literals'] modules= 3328 missing_key= 0 records= 4674
  by form: Counter({'argv': 4673, 'exec': 1})
  top programs: [('git', 507), ('--root', 203), ('127.0.0.1', 138), ('a', 96), ('bash', 94), ('session_id', 63), ('--repo-root', 57), ('REQ-001-001', 54), ('systemctl', 51), ('a.py', 48), ('--json', 44), ('tests', 44), ('U:Ox', 43), ('', 36), ('--uri', 35), ('uv', 34), ('config', 31), ('REQ-001-FR1-001', 30), ('--repo', 30), ('--check', 27), ('status', 26), ('check', 26), ('plan', 26), ('src', 24), ('list', 24), ('run', 24), ('python', 23), ('source', 23), ('init', 22), ('session', 20)]
plans/ast-tests.json captures= ['env_accesses', 'argv_literals'] modules= 2079 missing_key= 0 records= 4077
  by form: Counter({'argv': 4077})
  top programs: [('git', 354), ('--root', 203), ('127.0.0.1', 138), ('a', 96), ('bash', 87), ('session_id', 63), ('--repo-root', 57), ('REQ-001-001', 54), ('a.py', 48), ('systemctl', 46), ('tests', 44), ('--json', 43), ('U:Ox', 42), ('--uri', 35), ('config', 31), ('REQ-001-FR1-001', 30), ('--repo', 30), ('--check', 27), ('check', 26), ('plan', 26), ('status', 25), ('run', 24), ('source', 23), ('init', 22), ('src', 22), ('list', 21), ('add', 19), ('read', 18), ('cypher', 18), ('session', 17)]

Required invariants: captures == ['env_accesses', 'argv_literals'] in BOTH files ✓; missing_key == 0 in BOTH ✓.

JSON sizes before (step 0) / after (§7.5): plans/ast-ultimate.json 89,569,413 → 90,837,633 bytes; plans/ast-tests.json 60,514,609 → 61,436,691 bytes.

Measured facts worth carrying into Q2b/Q3:

• form distribution: 4,673 argv, 1 exec, 0 shell.
• The single exec record: src/orchestrata_ultimate/harness/evaluators/lint.py:65, asyncio.create_subprocess_exec, program ruff.
• 0 shell records is a true measurement, not a capture gap: the tree contains exactly two real shell-form call sites (tools/schedule_cron_tool.py:1176, agents/tool_registry.py:1498, both asyncio.create_subprocess_shell), and both pass a computed command (job.command, command) — skipped, not guessed, per the literals-only rule. There are no os.system/os.popen/getoutput call sites with literal strings.
• The over-captured tokens (--root, 127.0.0.1, REQ-001-001, '', …) are literal-headed lists passed to non-spawn callees — recorded verbatim with their site, for the Prolog rule to classify. Bare python appears 23 times, bash 94, uv 34, git 507.

§7.7 gate

gate: PASS
  - 5 rule clause(s), no violations
  fact base: 426,379 facts across 23 predicates
gate rc=0

Exactly the step-0 verdict. Fact count 426,176 → 426,379 (+203): the new test module's facts plus the facts of the new definitions §4 itself adds to src/orchestrata_ast (the ArgvLiteralInfo class, _analyze_argv_literals, module constants) — see §6 finding 2. No new facts from argv_literals themselves: this task adds snapshot fields, not facts or rules.

4. Verify (§8)

1. §8.1 focused tests — exact command from the spec, all six modules: 135 passed, 1 warning in 14.15s — 0 failed, 0 skipped. (The new module alone, §7.2: 15 passed.)

2. §8.2 staging — git add of exactly the five §5 paths; git status --porcelain directly after:

   M  plans/ast-tests.json
   M  plans/ast-ultimate.json
   M  src/orchestrata_ast/analyzer.py
   M  src/orchestrata_ast/models.py
   A  tests/test_orchestrata_ast_argv_literals.py
   ?? plans/briefs/qwen/
   

   plans/briefs/qwen/ left untracked, not staged, per the spec. Side effects the §8.4 run added AFTER staging (left unstaged and untouched, named here as §8.2 requires):

   • M reports/run-tests-latest.json — the run ledger scripts/run_tests.py writes on every run. Not in the §5 list; not mine to stage.
   • plans/ast-ultimate.json now shows MM: run_tests.py's freshness layer ([freshness] ast-runtime-merge: REFRESHED) re-injected "runtime_calls" enrichment into the working-tree copy (55,160 entries) after staging. The staged index still holds the §7.5 regeneration verbatim (0 runtime_calls), which is what §7.6/§7.7 measured and what §5 sanctions ("regenerated by §7 commands only"). See §6 finding 5 for the commit-time decision this leaves the orchestrator.
   • plans/reference-edges.json.affected-scratch — regenerated by the §8.4 resolver; gitignored (.gitignore:302 plans/*.affected-scratch), invisible to status.

3. §8.3 pre-commit review — scripts/gate.py review-staged:

   commit review: PASS
     3 review(s), no blocking findings
   review rc=0
   

4. §8.4 bounded run — resolver over the five §5 paths: "escalate": false, "untested": [], 34 test paths (the orchestrator's pre-dispatch 32 + tests/test_orchestrata_ast_argv_literals.py + tests/gates/test_commit_review.py, resolved via reference edges; spec expected ~33). Then ONE run of exactly those paths through scripts/run_tests.py: all 4 services + kernel verified live, all blocking safety gates pass, and:

   [run_tests] ✅ 837 tests passed, 0 failed, 0 errors, 0 skipped · 34/34 modules in 292s (4.9m)
   run rc=0
   

   .q2a-affected.json deleted afterwards (rm .q2a-affected.json). No second run started; no full-suite run; no xdist.

Lint/format (§7.3): ruff check --fix → "All checks passed!"; black → reformatted analyzer.py and the test file; tests re-run green after formatting; mypy on both src files: "Success: no issues found in 2 source files" (pre-check for §8.3, which passed).

5. STOP conditions

None hit. Every §7/§8 gate passed on the first attempt in the prescribed order.

6. Findings in the spec (§9: "anything in this spec you found wrong")

1. §4.2's argv (direct) table row contradicts §6 test 8. The row confines the direct scan to the "first positional argument", but test 8 requires asserting "the exec record AND the separate argv record" for os.execvp("gcc", ["gcc", "a.c"]), where the literal-headed list is the SECOND positional — "both shapes match" is only true if the argv shape is looked for beyond position 0. Resolution implemented: the direct argv scan checks EVERY positional argument for a literal-headed List/Tuple/BinOp-Add; the keyword fallback (args/cmd/command/argv, when there are no positionals) and every other row are unchanged. This matches §4.2's own over-capture prose ("the snapshot records every literal-headed argument list together with its callee") and the not-captured bullet ("a literal list that no call consumes" — implying a list a call consumes at any position is consumed). Consequence, stated plainly: os.spawnv(os.P_WAIT, "/usr/bin/gcc", ["gcc"]) yields the exec record program="/usr/bin/gcc" (as the spec says) PLUS an argv record program="gcc" for the third-positional literal list — the same phenomenon the spec explicitly annotates for execvp but leaves unmentioned for spawnv. The test asserts the full record set. Nothing else in §4 or §6 behaves differently under this reading; all 15 tests pass as specified.
2. §7.7's "the fact count changes only by the facts of your new test module" is imprecise. Measured +203 facts (426,176 → 426,379): the new test module contributes, and so do the new definitions §4 mandates inside src/orchestrata_ast (a new dataclass, a new method, module constants — all snapshot modules whose functions/classes are facts). Verdict requirements (PASS, 5 clauses, no violations, rc 0) hold exactly.
3. §8.4's module estimate (~33; pre-dispatch 32) resolved to 34 — the delta is the new test file itself and tests/gates/test_commit_review.py (reference-edge resolution over the regenerated snapshot). escalate false, untested empty, run green.
4. §8.2's "only those paths as staged (A /M )" cannot survive the spec's own ordering (stage in §8.2, then run §8.4): run_tests.py writes reports/run-tests-latest.json and its freshness layer re-enriches the working-tree plans/ast-ultimate.json (runtime_calls), so the final status is MM for that one path plus M for the ledger. Handled per §8.2's own rule for changes not made by me: left unstaged and untouched, named here. The staged index is the §7 deliverable.
5. runtime_calls lifecycle note for the commit decision (not a defect, but material): the committed origin/master snapshot carries 55,049 runtime_calls entries; the §7-mandated ast-sync --snapshot-only writes the snapshot WITHOUT them (the step-0 baseline regeneration had the same property); run_tests.py's ast-runtime-merge freshness layer re-injects them (55,160) into the working tree on the next run. Committing the staged index therefore ships a snapshot whose runtime_calls enrichment returns on the first run after commit. Both states pass gate.py run (step-0 baseline and §7.7 were measured on unmerged files). The choice between committing the index as staged or re-adding the merged working tree belongs to the orchestrator.

7. §9 done-when checklist

• [x] §6 tests 1–15 exist (one function each) and pass; §8.1 module set: 135 passed, 0 skipped.
• [x] Snapshot: captures == ['env_accesses', 'argv_literals'] and missing_key == 0 in BOTH JSON files; first regen = fallback (full reparse) (128 s), second = warm — reused 3328, reparsed 0, removed 0 (13 s).
• [x] gate.py run = gate: PASS, 5 rule clause(s), no violations, rc 0 — the step-0 baseline; new fact count 426,379.
• [x] gate.py review-staged rc 0 (commit review: PASS, 3 review(s), no blocking findings) with only the §5 files staged.
• [x] §8.4 resolver: escalate false, untested []; run_tests.py <34 resolved paths>: 837 passed, 0 failed, 0 errors, 0 skipped, rc 0; .q2a-affected.json deleted.

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks