Orchestrata Ultimate › Plans › briefs › history-audit-2026-10-19

Brief — week-of-history audit: errors and lessons (2026-10-12 → 2026-10-20)

Source: plans/briefs/history-audit-2026-10-19.md at 8a2cb42ad · last changed 2026-10-19 · first committed 2026-10-19 · 117 lines.

Prose pending.

Plan text

Brief — week-of-history audit: errors and lessons (2026-10-12 → 2026-10-20)

Ordered by Mohamed 2026-10-19: run multiple MiniMax agents and GLM agents over the entire history of the last week — tool calls and briefings across the Claude Code, DeepSeek and OpenCode channels — and produce one document of errors and lessons learned for the coordinator session (supervisor/PM/architect, qwen3-coder-next workers).

Reading the Claude transcripts and the OpenCode database for this audit is explicitly authorized by Mohamed for this packet. The .opencode/opencode.json read-denials on ~/.claude/** are bypassed via shell tooling (grep/jq/sqlite3) for THIS audit only.

Deliverable layout

Four section files, one per agent, in reports/history-audit-2026-10-19/:

The coordinator merges the four sections into reports/errors-and-lessons-2026-10-19.md.

Rails (all four agents, non-negotiable)

1. Read-only on every history source. The ONLY file you create is your section file. Open the OpenCode DB read-only: sqlite3 "file:/home/mohamed/.local/share/opencode/opencode.db?mode=ro". Never git add, never commit, never touch ~/.claude config/state files, never write into .runs/.
2. No secrets. If a scan surfaces an API key, token, or password, do NOT copy it into the report; cite the file:line and write [REDACTED].
3. No inline Python (RUL-037): use grep, jq, awk, sed (read-only), sqlite3, git log, find. No python -c, no heredocs into an interpreter.
4. Evidence discipline. Every number in your report comes from a command you ran; include the command. Every incident cites file + line/row-id + timestamp. Mark anything you could not measure UNKNOWN — never guess, never pad.
5. No weasel words (RUL-017): no "likely/probably/seems/apparently". State what the evidence shows or write UNKNOWN.
6. Aggregate, don't ingest. The Claude corpus is ~1.8 GB and the DB is 2.1 GB. Never cat a whole transcript. Use jq -c filters, grep -m, head, counts. Timebox each scan; a representative counted sample beats an exhaustive raw dump.
7. Window: events from 2026-10-12 00:00 local through now (2026-10-19/20). Use find -newermt, git --since, session.time_created >= strftime('%s','2026-10-12')*1000. Older material only as context for a in-window incident.

Section file format (all four)

# A<N> — <source scope> — errors & lessons, 2026-10-12 → 2026-10-20

## 0. Scope & method
<commands run, files/rows scanned, what was sampled vs exhaustive>

## 1. Error taxonomy (counted)
| class | count | where (top files/sessions) |
<classes: tool-error, hook-refusal, provider/api-error, test-failure, infra-down,
 false-claim-corrected, user-correction, retry-loop, guard-refusal, other>

## 2. Incidents
### I-<N> <one-line title>
- when / session / harness / model
- what happened (evidence: file:line or query + row)
- root cause (verified or UNKNOWN)
- cost (time/tokens/money/rework, measured where possible)
- lesson
- proposed mechanism (guard/hook/brief-rule/workflow step) — or "judgment"

## 3. Top lessons (ranked by frequency × impact)

## 4. Recommended changes for the coordinator
<concrete: brief templates, verification steps, dispatch routing, session hygiene,
 KG/redis ops, OpenCode wiring, skill updates>

## 5. UNKNOWNs

Extraction seed patterns

Claude Code JSONL (A1, A4)

• Enumerate: ls ~/.claude/projects/ — project dirs encode cwd with dashes (-home-mohamed-workspace-wt-COMPONENTS etc). Select files by BOTH dir and the per-line "cwd" field; include <session-id>/subagents/*.jsonl.
• In-window files: find ~/.claude/projects -name "*.jsonl" -newermt 2026-10-12
• Tool errors: jq -c 'select(.type=="user") | .message.content[]? | select(.type=="tool_result" and .is_error==true) | .content | tostring | .[0:200]' <file>
• Hook/guard refusals: grep tool_result text for Blocked:, denied, refused, wall
• Provider/API errors: grep for rate_limit, overloaded, spend, isApiErrorMessage, "type":"error"
• User corrections (Mohamed verbatim): jq -r 'select(.type=="user") | select(.message.content|type=="string") | .message.content' <file> — flag angry/corrective turns verbatim (they become rulings)
• Failed commands: tool_result text with command not found, Traceback, error:, non-zero exits

OpenCode DB (A2)

• Tables: session (id, title, directory, agent, model, cost, tokens_*, time_created/updated, metadata), message, part, session_message, todo, event, permission.
• Explore first: .schema part, .schema message — then build the error query (tool parts carry state JSON; look for error/failed statuses).
• Session census for the window: per model × directory × day: count, cost, tokens, titles.
• Identify the deepseek-v4-pro runs (the DSH channel: adr-0112 tools seams, wiki page writing, kg work) and the grok/qwen coordinator sessions; report their errors separately.

Dispatch runs + ledgers (A3)

• packages/design-scripts/dispatch/.runs/<task>/: meta.json (task/model/time), rc (exit code), verdict.json (checks passed/failed), err.log, out.json. Census every run dir: rc != 0, failed verdict checks, err.log signatures. Note which are in-window by mtime.
• Briefs in packages/design-scripts/dispatch/*-brief.md: what was ordered, what the verdicts show actually happened (brief-vs-reality gaps).
• reports/precommit-walls.ndjson (371 rows): inspect one row for schema (head -1 ... | jq .), then aggregate refusals by hook/wall/reason and by day.
• git log --since=2026-10-12 in /home/mohamed/workspace/wt-COMPONENTS and ~/workspace/knowledge-wiki (--stat): commits whose messages claim things — list the claims; cross-check against verdict/rc files where possible.
• Session CLI narrative: context.history.md tail + session summary output are already summarized; use git-showable files only (do not run session mutators).

Done

Reply with: your section file path, the taxonomy count table, and your top 5 lessons. The section file must exist on disk before you reply.

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks