Orchestrata Ultimate › Plans › codemod-engine-briefs › W1-AGENT-CONTEXT

W1 Agent Context Document

Source: plans/codemod-engine-briefs/W1-AGENT-CONTEXT.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-27 · 600 lines.

Prose pending.

Plan text

W1 Agent Context Document

Wave: W1 (Forest + Sentinel)
Date: 2026-07-27
Status: Ready for Implementation

1. Complete Standards Reference

1.1 Architecture Rules (NEVER VIOLATE)

Component Boundaries

Hard Rules

1. ⛔ Gateway NEVER serves HTML/JS/CSS. Adding process_request for static files is an architectural violation.
2. ⛔ VizInt is independent of the Gateway. Browser loads UI from port 5000.
3. ⛔ Never use except: pass or except Exception: pass. Log actual error.
4. ⛔ gRPC kernel is out-of-process. Gateway connects via gRPC to kernel on port 50053.
5. ⛔ Never hardcode service IPs or ports. All from .env.
6. ⛔ Never dismiss a test failure. Trace every failure to root cause.
7. ⛔ The kernel is the beating heart — REQUIRED for every suite run.
8. ⛔ Built is not done — WIRED is done. Every step's definition-of-done includes a wiring check.

1.2 Test Standards (Non-Negotiable)

Definition of Done: ≥90% coverage (Non-Negotiable)

• Measure per file: uv run pytest tests/<path> --cov=<the.module.path> --cov-report=term-missing -o addopts=""
• Tests must be REAL — meaningful assertions on actual behaviour, using real infrastructure (no mocks/stubs)
• When a test fails, fix the CODE — never fake the test green.

9 Mandatory Transformations for Ported Tests

General Test Rules

• No mocking — real implementations only (no unittest.mock, MagicMock, AsyncMock, patch)
• No stubs or fakes — no FakeX, StubX, DummyX classes
• No monkeypatch — pass config/parameters explicitly
• No skips — no @pytest.mark.skip, @pytest.mark.skipif
• ⛔ AGENT RULE: Never skip, ignore, or dismiss a test failure
• Full parallel isolation — every test runs under the batched executor, never pytest -n auto
• UUID-prefixed resources — test_<UUID>_<name> pattern
• Function-scope fixtures — no module/class scope for mutable state

1.3 @traces Conventions

Enforced deterministically by trace-lint:

• No dangling ids (must exist in some specs/*/checklists/requirements.md)
• No REQ-000-000 placeholders
• No malformed ids
• No unknown spec= values
• No spec-mismatch (every req id must be defined by the spec the decorator claims)
• ≤25 reqs per decorator (>10 warns)

Judgment Conventions

• Two-lane rule: Business-facing code traces to FRs. Infrastructure code traces only to explicit NFR/architectural requirements.
• Sparse-and-accurate beats padded. A def with no genuinely-matching requirement stays untraced.
• Class vs method. Class decorators carry the few reqs describing the class's collective responsibility; method-specific reqs go on methods.
• Mixed provenance = stacked decorators. Ids from two specs never share one decorator.

1.4 Code Style Standards

Import Rules

• Absolute imports only — no relative imports (from . import X)
• Import order: stdlib → third-party → first-party → local
• No wildcard imports — no from X import *

Exception Handling

• Never use except: pass — always log the actual error
• Never use except Exception: pass — always log the actual error
• Use specific exception types — catch what you expect

Environment Variables

• Never use os.environ["X"] — always use os.environ.get("X", default)
• Never hardcode service IPs or ports — all from .env

Type Hints

• All public functions must have type hints — parameters and return type
• Use from __future__ import annotations for forward references
• Use typing.Protocol for interfaces — not ABC

1.5 Performance Standards

Parallel Execution

• Use ThreadPoolExecutor for CPU-bound work — Python 3.14.6 free-threaded (GIL disabled)
• 12 CPU cores available — exploit parallelism
• Never use pytest-xdist — deadlocks ~78% of the time

Memory Management

• Measure RSS delta — use psutil.Process().memory_info().rss
• Target: <3 GiB RSS delta for full codebase load (2,500 files, 780K LOC)

1.6 Security Standards

File Operations

• Use O_NOFOLLOW flag — prevent symlink attacks
• Use fcntl.flock() for advisory locking — prevent concurrent writes
• Atomic writes — tempfile + fsync + rename

Input Validation

• Validate all inputs — never trust external data
• Use path confinement — all paths must be within repo root

1.7 Documentation Standards

Docstrings

• All public classes and functions must have docstrings
• Use Google style — Args, Returns, Raises sections
• Include examples — show how to use the function

Comments

• Explain WHY, not WHAT — code should be self-documenting
• No TODO comments — either implement it or create an issue

2. Available Tools and Gates

2.1 Checking Scripts

2.2 AST Analysis Tools

2.3 Test Execution Tools

2.4 Linting Tools

2.5 Gate Commands (Must Pass Before Every Commit)

# 1. Linting
.venv/bin/ruff check src tests scripts
.venv/bin/black --check src/orchestrata_ultimate/refactor/codemod tests/refactor/codemod scripts/codemod.py

# 2. Standards
PYTHONPATH=src .venv/bin/python scripts/serialize_codebase_ast.py standards

# 3. Dependencies
PYTHONPATH=src .venv/bin/python scripts/serialize_codebase_ast.py check-deps --include-tests

# 4. Test review
PYTHONPATH=src .venv/bin/python scripts/test-review.py tests/refactor/codemod/ --stdout

# 5. Type checking
PYTHONPATH=src .venv/bin/python scripts/pyright_ratchet.py --max 283

2.6 Verification After Implementation

# ALL of these must return 0 matches:
grep -rn 'os\.environ\[.*\] =' tests/refactor/codemod/ --include="*.py"
grep -rn 'scope="module"\|scope="class"' tests/refactor/codemod/ --include="*.py"
grep -rn '^[a-zA-Z_].*= \[\|^[a-zA-Z_].*= {}\|^[a-zA-Z_].*= None' tests/refactor/codemod/conftest.py
grep -rn 'port=5[0-9]\{4\}' tests/refactor/codemod/ --include="*.py"
grep -rn 'monkeypatch' tests/refactor/codemod/ --include="*.py"
grep -rn 'MagicMock\|AsyncMock\|Mock\|patch(' tests/refactor/codemod/ --include="*.py"
grep -rn 'Fake\|Stub\|Dummy' tests/refactor/codemod/ --include="*.py"
grep -rn 'pytest\.mark\.skip' tests/refactor/codemod/ --include="*.py"

# Must pass under parallel isolation:
python scripts/run_tests.py tests/refactor/codemod/ --skip-gates

3. W1 Specific Requirements

3.1 FOREST Agent (Qwen Max)

Files to Implement

1. src/orchestrata_ultimate/refactor/codemod/workspace.py (~400 LOC)

   • CodemodWorkspace class
   • load() classmethod (eager parse, parallel on 12 cores)
   • run() method (execute transform passes)
   • commit() method (review + write-back)
   • forest property (dict of path → libcst.Module)
   • unparsable property (list of failed files)
   • stats() method (WorkspaceStats)

2. src/orchestrata_ultimate/refactor/codemod/_csr_index.py (~200 LOC)

   • _CSRIndex class
   • __init__() (build CSR from edge list)
   • dependencies() (forward neighbors)
   • dependents() (reverse neighbors)
   • transitive_closure() (BFS with depth limit)

3. src/orchestrata_ultimate/refactor/codemod/_cig.py (~500 LOC)

   • CodeIntelligenceGraph class
   • dependencies(), dependents() (O(1) via CSR)
   • blast_radius() (BFS transitive closure)
   • breaking_changes() (signature diff + downstream consumers)
   • cycles() (Tarjan's SCC algorithm)
   • topological_order() (Kahn's algorithm)
   • affected_tests() (reverse TESTED_BY edges)
   • coupling_score() (efferent/afferent + instability)
   • abstraction_level() (Protocol/ABC ratio)
   • centrality() (betweenness centrality)
   • impact_score() (PageRank-style)

4. src/orchestrata_ultimate/refactor/codemod/_cig_builder.py (~300 LOC)

   • build_cig() function (extract nodes and edges from CST forest)
   • Node extraction (Module, Class, Function, Method, Parameter, Test, Import)
   • Edge extraction (IMPORTS, CALLS, INHERITS, DEFINES, TESTED_BY, OVERRIDES)

Files to Test

1. tests/refactor/codemod/test_workspace_load.py (~300 LOC)

   • 7 BDD scenarios from workspace_load.feature
   • Test eager parse (serial and parallel)
   • Test lossless round-trip
   • Test unparsable files
   • Test RSS delta

2. tests/refactor/codemod/test_workspace_index.py (~200 LOC)

   • 6 BDD scenarios from workspace_index.feature
   • Test modules() mapping
   • Test dependencies() and dependents()
   • Test symbol_table()

3. tests/refactor/codemod/test_code_intelligence_graph.py (~400 LOC)

   • 10 BDD scenarios from code_intelligence_graph.feature
   • Test all CIG query methods
   • Test performance (<2ms per query)

4. tests/refactor/codemod/test_csr_index.py (~200 LOC)

   • Plain pytest tests (not BDD)
   • Test CSR construction
   • Test forward/reverse lookups
   • Test transitive closure
   • Test edge cases (empty graph, self-loops, disconnected components)

Performance Targets

• Parallel parse: <120s for 2,500 files (12 cores)
• Serial parse: <360s for 2,500 files
• RSS delta: <3 GiB
• CIG query latency: <2ms per query
• CSR lookup: O(1)

Success Criteria

• [ ] _csr_index.py implemented with O(1) lookups
• [ ] _cig_builder.py extracts all node and edge types
• [ ] _cig.py implements all 11 query methods
• [ ] workspace.py loads codebase in <120s (parallel)
• [ ] All BDD scenarios pass (workspace_load, workspace_index, code_intelligence_graph)
• [ ] CSR index tests pass (plain pytest)
• [ ] Performance targets met (<2ms per CIG query)
• [ ] All gates pass (ruff, black, standards, check-deps, test-review, pyright_ratchet)
• [ ] Coverage ≥90% per file

3.2 SENTINEL Agent (Sonnet)

Files to Implement

1. src/orchestrata_ultimate/refactor/codemod/txn.py (~250 LOC)
   • commit_changes() function
   • _snapshot_files() (capture pre-commit bytes)
   • _write_atomic() (tempfile + fsync + rename)
   • _batch_fsync() (fsync directory fds)
   • _verify_post_write() (re-parse written files)
   • _rollback() (restore from snapshot)
   • _check_concurrent_edits() (mtime + hash + O_NOFOLLOW + flock)

Files to Test

1. tests/refactor/codemod/test_transactional_writeback.py (~300 LOC)
   • 6 BDD scenarios from transactional_writeback.feature
   • Test atomic write (tempfile + fsync + rename)
   • Test batch fsync (5 dirs instead of 100 files)
   • Test post-write verification
   • Test rollback on failure
   • Test concurrent-edit guard (mtime + hash)
   • Test symlink attack prevention (O_NOFOLLOW)
   • Test advisory locking (fcntl.flock)

Security Requirements

• Atomic writes — tempfile + fsync + rename
• Batch fsync — fsync directory fds (not per-file)
• Concurrent-edit guard — mtime + content hash (xxhash3_64)
• Symlink attack prevention — O_NOFOLLOW flag
• Advisory locking — fcntl.flock(LOCK_EX | LOCK_NB)
• Rollback on failure — restore from snapshot

Success Criteria

• [ ] txn.py implements atomic writes with rollback
• [ ] Batch fsync reduces I/O overhead (5 dirs vs 100 files)
• [ ] Concurrent-edit guard detects modifications
• [ ] Symlink attacks prevented (O_NOFOLLOW)
• [ ] Advisory locking prevents concurrent writes
• [ ] All BDD scenarios pass (transactional_writeback)
• [ ] All gates pass (ruff, black, standards, check-deps, test-review, pyright_ratchet)
• [ ] Coverage ≥90% per file

4. Skeleton Code (W0 Deliverables)

The skeleton code has been generated and split into proper modules:

src/orchestrata_ultimate/refactor/codemod/
├── __init__.py (public API exports)
├── report.py (ChangeEntry, ChangeSet, Finding, CommitReport)
├── workspace.py (CodemodWorkspace)
├── _csr_index.py (CSR adjacency arrays)
├── _cig.py (CodeIntelligenceGraph)
├── passes.py (TransformPass + 5 built-in passes)
├── review.py (ReviewPass + 4 review checks)
├── daemon.py (CodemodDaemon)
├── server.py (CodemodServer)
├── client.py (CodemodClient)
├── watcher.py (FileWatcher)
├── _types.py (type stubs)
├── observability/
│   ├── events.py (CodemodEvent, CodemodEventType)
│   ├── tracing.py (TracingObserver)
│   ├── metrics.py (MetricsObserver)
│   ├── debug_panel.py (DebugPanelObserver)
│   ├── error_detector.py (ErrorDetector)
│   └── graph_writer.py (GraphWriterObserver)
└── persistence/
    └── repository.py (CodemodGraphRepository, BoltSyncCodemodRepository)

Skeleton Verification Results

All skeleton code has been verified:

• ✅ 19 Python files generated (1,649 total lines)
• ✅ All imports successful (public API + internal modules)
• ✅ All ruff checks pass (0 errors)
• ✅ All black formatting applied (19 files formatted)
• ✅ 112 # IMPLEMENT: markers (locations where agents add logic)
• ✅ 112 raise NotImplementedError() stubs (matching IMPLEMENT markers)
• ✅ Type stubs in _types.py (Module, CIGNode, BreakingChange, CodeChange, WorkspaceEditPlan, SyncGraphRepository, CodemodRun, CodemodPass, CouplingScore, FindingSeverity, ParallelismType, CodemodEventType, Observer)

Skeleton Structure

All skeleton files have:

• # IMPLEMENT: comments marking where to add logic
• raise NotImplementedError() stubs
• Type hints for all parameters and return types
• Docstrings for all public classes and methods
• Proper imports from _types.py and report.py

Import Structure

# Public API (from __init__.py)
from orchestrata_ultimate.refactor.codemod import (
    ChangeEntry, ChangeSet, Finding, CommitReport,
    CodemodEvent, CodemodEventType,
    CodemodWorkspace, CodeIntelligenceGraph,
    TransformPass, ReviewPass,
)

# Internal modules
from orchestrata_ultimate.refactor.codemod._csr_index import _CSRIndex
from orchestrata_ultimate.refactor.codemod._cig import CodeIntelligenceGraph
from orchestrata_ultimate.refactor.codemod._types import Module, CIGNode, BreakingChange
from orchestrata_ultimate.refactor.codemod.report import ChangeEntry, ChangeSet, Finding, CommitReport
from orchestrata_ultimate.refactor.codemod.workspace import CodemodWorkspace
from orchestrata_ultimate.refactor.codemod.passes import TransformPass, ReplaceStringLiteralPass
from orchestrata_ultimate.refactor.codemod.review import ReviewPass, ParseIntegrity
from orchestrata_ultimate.refactor.codemod.daemon import CodemodDaemon
from orchestrata_ultimate.refactor.codemod.server import CodemodServer
from orchestrata_ultimate.refactor.codemod.client import CodemodClient
from orchestrata_ultimate.refactor.codemod.watcher import FileWatcher
from orchestrata_ultimate.refactor.codemod.observability.events import CodemodEvent, CodemodEventType, Observer
from orchestrata_ultimate.refactor.codemod.observability.tracing import TracingObserver
from orchestrata_ultimate.refactor.codemod.observability.metrics import MetricsObserver
from orchestrata_ultimate.refactor.codemod.observability.debug_panel import DebugPanelObserver
from orchestrata_ultimate.refactor.codemod.observability.error_detector import ErrorDetector
from orchestrata_ultimate.refactor.codemod.observability.graph_writer import GraphWriterObserver
from orchestrata_ultimate.refactor.codemod.persistence.repository import CodemodGraphRepository, BoltSyncCodemodRepository

5. Dependencies (Already Installed)

• ✅ libcst>=1.8.6 (CST parsing with native Rust parser)
• ✅ xxhash>=3.0.0 (content fingerprinting, 5 GB/s)
• ✅ psutil>=7.2.2 (RSS measurement)

6. Test Infrastructure

6.1 BDD Configuration

• Feature files: specs/110-codemod-engine/features/
• Step definitions: tests/refactor/codemod/conftest.py (shared) + per-test-file (specific)
• Scenario binding: @scenario per scenario (explicit, with TST/REQ docstrings)
• Tags: @REQ_110_NNN, @TST_110_NNN, @FR_NNN on scenarios

6.2 Fixtures

# tests/refactor/codemod/conftest.py

@pytest.fixture
def codemod_repo(tmp_path: Path) -> Path:
    """Real git repository under tmp_path with src/ layout."""
    subprocess.run(["git", "init", "-q"], cwd=tmp_path, check=True)
    subprocess.run(["git", "config", "user.email", "test@example.com"], cwd=tmp_path, check=True)
    subprocess.run(["git", "config", "user.name", "Test"], cwd=tmp_path, check=True)
    return tmp_path

@pytest.fixture
def fixture_package(codemod_repo: Path) -> Callable[[Mapping[str, str]], Path]:
    """Factory: write src/-rooted files, return repo root."""
    def make(files: Mapping[str, str]) -> Path:
        src_root = codemod_repo / "src"
        for relative, text in files.items():
            target = src_root / relative
            target.parent.mkdir(parents=True, exist_ok=True)
            target.write_text(text, encoding="utf-8")
        return codemod_repo
    return make

6.3 Shared Step Definitions

# tests/refactor/codemod/conftest.py

from pytest_bdd import given, when, then, parsers

@given(parsers.parse("the fixture package contains {count:d} Python files"), target_fixture="repo_root")
def fixture_with_files(fixture_package, count):
    files = {}
    for i in range(count):
        files[f"pkg/mod_{i}.py"] = f"# Module {i}\nx_{i} = {i}\n"
    files["pkg/__init__.py"] = ""
    return fixture_package(files)

@when("I load the workspace", target_fixture="workspace")
def workspace_loaded(repo_root):
    return CodemodWorkspace.load(repo_root, include=("src",))

@then(parsers.parse("the forest contains exactly {count:d} modules"))
def forest_count(workspace, count):
    assert len(workspace.forest) == count

7. Execution Workflow

7.1 FOREST Agent Workflow

1. Read skeleton files — understand the structure
2. Implement _csr_index.py — CSR adjacency arrays
3. Implement _cig_builder.py — extract nodes and edges from CST
4. Implement _cig.py — all 11 query methods
5. Implement workspace.py — load, run, commit
6. Write tests — BDD scenarios + plain pytest
7. Run gates — all must pass
8. Verify coverage — ≥90% per file
9. Commit — with proper message

7.2 SENTINEL Agent Workflow

1. Read skeleton files — understand the structure
2. Implement txn.py — atomic writes with rollback
3. Write tests — BDD scenarios
4. Run gates — all must pass
5. Verify coverage — ≥90% per file
6. Commit — with proper message

7.3 Integration Workflow

1. Merge results — both agents write to same directory
2. Run all W1 tests — verify integration
3. Fix any issues — import errors, type mismatches
4. Run all gates — all must pass
5. Commit — with proper message

8. Common Pitfalls

8.1 Test Failures

• Never dismiss a test failure — trace to root cause
• Never use pytest.skip() — fix the test or the code
• Never use mocks — use real infrastructure

8.2 Performance Issues

• Use ThreadPoolExecutor for parallel work — Python 3.14.6 free-threaded
• Measure RSS delta — use psutil.Process().memory_info().rss
• Use CSR for O(1) lookups — not dict-of-sets

8.3 Security Issues

• Use O_NOFOLLOW flag — prevent symlink attacks
• Use fcntl.flock() for locking — prevent concurrent writes
• Validate all paths — must be within repo root

8.4 Type Checking Issues

• Use from __future__ import annotations — for forward references
• Use typing.Protocol for interfaces — not ABC
• All public functions must have type hints — parameters and return type

9. References

• SPEC-110: specs/110-codemod-engine/spec.md
• Design Document: plans/codemod-engine-design.md
• Execution Plan: plans/codemod-engine-execution-plan.md
• BRIEF-W1: plans/codemod-engine-briefs/BRIEF-W1.md
• AGENTS.md: AGENTS.md (complete standards reference)
• Feature Files: specs/110-codemod-engine/features/
• Skeleton Code: src/orchestrata_ultimate/refactor/codemod/

10. Contact

If you encounter any issues or have questions:

1. Read the documentation — SPEC-110, design doc, AGENTS.md
2. Run the gates — they will catch most issues
3. Check the skeleton code — it has the correct structure
4. Ask for help — don't guess, ask for clarification

Good luck! You have all the tools and standards you need to succeed.

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks