Orchestrata Ultimate / Specs / 109-rename-engine

109-rename-engine: Deterministic LSP-Grade Rename Engine

A deterministic rename engine: it renames Python symbols by type resolution (not by name matching), moves modules atomically, executes declarative rename plans, and emits a machine-readable residue report of every occurrence it matched but deliberately did not rewrite.

At a glance

Reading this spec

This spec defines a deterministic rename engine that renames Python symbols by type resolution rather than by name matching. The motivating incident is stated in the text: LLM agents half-apply mechanical renames, and a gw to slot label rename that missed four Redis slot parsers produced 54 test failures. The existing tooling cannot prevent this, because the in-repo AST rename matches name strings and cannot tell two classes with the same name apart, and the LSP tool of spec 065 has no JSON-RPC layer. The one piece with the right semantics is the fail-closed type-scoped method rename, and this spec generalizes that contract to every rename kind.

The engine has three capabilities. Symbol rename uses a real basedpyright-langserver textDocument/rename round trip and disambiguates same-named symbols by defining module. Module move runs git mv, rewrites every absolute import and rewrites string and path literals it can prove, refusing on ambiguity. Rename plan executes a declarative TOML or JSON plan atomically. The design treats the language server's WorkspaceEdit as the authority on what to rewrite and an independent name-based scan as the candidate oracle; the difference between the two is a machine-readable residue report of every occurrence matched but deliberately not rewritten. A residue item that is a resolvable code reference is a hard error that aborts the rename. Non-goals are other languages, semantic renaming inside Prolog rule bases and Cypher strings (reported as residue), and replacing the existing AST tool.

Five user stories cover renaming one of two same-named classes, the engine refusing rather than half-applying, moving a module atomically, executing a plan, and a volk renaming through the tool registry. The 62 requirement IDs (REQ-109-NNN) include: the engine opens the defining document before prepareRename; it aborts when cross-file candidates exist but the edit touches only the defining file; modules move with git mv; a non-provable occurrence is residue and never rewritten; execution snapshots, applies, verifies and restores on failure; execute never raises and returns a failed ToolResult; an unknown target refuses and lists near-miss candidates; and a plan validation failure refuses before execution, naming the entry index.

No code module cites this spec, and its status line reads Draft. The requirements traceability matrix lists every ID. It tracks 2 files. No src/ symbol cites any of the 62 requirements, so measured trace coverage is 0.0%, and in tests/, 0 requirements are cited by a test-side @traces. All 62 requirements have no citing symbol; untraced means no @traces names the ID, and the behaviour may still exist in code that was never annotated.

Requirements

Checkbox is the state recorded in the requirements checklist; Traced by is the count of src/ symbols whose @traces names the ID.

Code that traces this spec

No src/ symbol cites a requirement of this spec.

Checklists

checklists/requirements.md

The requirement list above comes from this file; 0 of 62 checkbox items are ticked.

Full spec text

Verbatim from specs/109-rename-engine/spec.md; headings are pushed down two levels.

SPEC-109: Deterministic LSP-Grade Rename Engine

Metadata

• Spec ID: 109-rename-engine
• Title: Rename Engine — Type-Resolved Symbol Rename, Atomic Module Move, and Declarative Rename Plans
• Author: Mohamed
• Status: Draft
• Created: 2026-07-27
• Last Updated: 2026-07-27
• Priority: P1
• Version: 0.1.0
• Extends: SPEC-103 (Tool-Calling Refactor), SPEC-072 (Tool Registry), SPEC-065 (LSP Tool)

File layout note. The originating brief named this file specs/SPEC-109-rename-engine.md. It is placed at specs/109-rename-engine/spec.md instead, matching every other spec in the repo, because scripts/serialize_codebase_ast.py check-traces scans specs/*/checklists/requirements.md. A top-level spec file would make every REQ-109-* trace resolve as dangling.

MOVED 2026-10-20 (MOVE-109): This spec has moved to codetools. The rename engine now lives in codetools.refactor.rename.*. The engine files were copied to src/codetools/refactor/rename/ with decoupling edits (see reports/move-109-report-20261020.md). @traces decorators were stripped because codetools.rtm.traces cites the CODETOOLS spec registry; REQ-109 ids remain historically valid in this spec but declarations leave orchestrata code. A renumbering was not done to avoid updating all REQ-109 references across the codebase.

Overview

What

A deterministic rename engine: it renames Python symbols by type resolution (not by name matching), moves modules atomically, executes declarative rename plans, and emits a machine-readable residue report of every occurrence it matched but deliberately did not rewrite.

Three capabilities:

1. Symbol rename — type-resolved, via a real basedpyright-langserver textDocument/rename round-trip; disambiguates same-named symbols by defining module.
2. Module move — git mv plus rewrite of every absolute import, plus rewrite of string/path literals it can prove; refuses on ambiguity.
3. Rename plan — a declarative TOML/JSON plan executed atomically, producing a residue report for everything left for human/LLM judgment.

Why

LLM agents half-apply mechanical renames. The motivating incident: a gw → slot label rename missed four Redis slot parsers and produced 54 test failures. The existing in-repo tooling cannot prevent this:

• scripts/serialize_codebase_ast.py rename matches name strings on AST tokens. Its scope handling is a symtable line-range approximation (_shadow_line_ranges), its definition sites are located by regex (_def_name_span), and --kind method|attr rewrites every ast.Attribute whose .attr matches. It cannot distinguish two classes that share a name.
• src/orchestrata_ultimate/tools/lsp_tool.py (SPEC-065) has no JSON-RPC layer at all — its own docstring records that a real LSP client "was rejected for this MVP", and that FR-009 rename is not implemented. Its PythonSymbolIndex is ast.walk name-string matching.

Exactly one piece of in-repo code already has the right semantics: _type_scoped_method_rename (serialize_codebase_ast.py), which is fail-closed — if any receiver type is undeterminable it sets blocked=True and refuses --apply. SPEC-109 generalizes that contract to every rename kind and backs it with a real type checker.

How

Drive basedpyright-langserver --stdio (MIT, PyPI-distributed, self-contained — open source, unlike Pylance) over stdio JSON-RPC. Treat its WorkspaceEdit as the authority on what to rewrite, and an independent name-based occurrence scan as the candidate oracle. The difference between the two sets is the residue report. A residue item that is a resolvable code reference is a hard error that aborts the rename — never a silent skip.

Non-goals

• Renaming across languages (TypeScript, Prolog). Python only.
• Semantic renaming inside Prolog .pl rule bases or Cypher query strings — these are reported as residue, never guessed.
• Replacing scripts/serialize_codebase_ast.py. That tool is untouched by this spec; SPEC-109 ships alongside it.

Spike Findings (load-bearing — the design depends on these)

A live de-risk spike ran basedpyright 1.39.9 (based on pyright 1.1.411) against both a synthetic fixture and the real 122K-LOC tree. Full transcripts: plans/rename-engine-design.md § Spike. Four findings are normative:

Verdict: PASS. Type-resolved rename with same-name disambiguation works, and the two silent-under-rename failure modes are both detectable and preventable.

User Scenarios

US-109-001: Rename one of two same-named classes

As a maintainer I want to rename Neo4jConfig in knowledge_graph/config.py without touching the unrelated Neo4jConfig in orchestrator/neo4j_audit.py So that a mechanical rename cannot corrupt an unrelated subsystem Acceptance Criteria:

• The rename is targeted by defining module + symbol name, not by name alone.
• Every genuine reference in src/ and tests/ is rewritten.
• No occurrence belonging to the twin symbol is rewritten.
• Docstring/comment/string occurrences are reported as residue, not rewritten.

US-109-002: The engine refuses rather than half-applies

As an operator I want the engine to abort when it cannot see the whole picture So that I never get the 54-failing-test outcome Acceptance Criteria:

• If the type checker cannot resolve the target module's imports, the run aborts with a named error before any file is written.
• If a candidate occurrence is a resolvable code reference the type checker did not rewrite, the run aborts.
• A dry run is the default; writing requires an explicit --apply.

US-109-003: Move a module atomically

As a developer I want to move a/b.py to c/d.py and have every import follow So that the tree stays import-clean in one step Acceptance Criteria:

• The file moves via git mv (history preserved).
• Every absolute import of the old dotted path is rewritten.
• String literals that provably denote the old path (e.g. "a/b.py", "a.b" as a module string) are rewritten; ambiguous ones are refused.
• On any failure the whole move rolls back.

US-109-004: Execute a declarative rename plan

As a maintainer running a rename wave I want to declare all symbol/module/string renames in one file and execute it So that the wave is reviewable, repeatable, and atomic Acceptance Criteria:

• The plan validates before anything executes.
• Execution is all-or-nothing across every entry.
• A machine-readable residue report lists file:line, occurrence text, and reason for everything matched-but-not-executed.

US-109-005: A volk renames through the tool registry

As a volk I want a refactor_rename tool So that an agent can request a rename and receive the residue for judgment Acceptance Criteria:

• The tool conforms to the SPEC-072 BaseTool contract and never raises.
• It defaults to dry-run and is marked requires_approval.
• Its ToolResult.output carries the plan and residue as JSON.

Data Models

All models live in refactor/rename/models.py, are frozen dataclasses, and are contract-frozen by this spec so parallel builders can code against them.

class ResidueReason(StrEnum):
    STRING_LITERAL   = "string_literal"    # inside a str/bytes literal
    DOCSTRING        = "docstring"
    COMMENT          = "comment"
    OTHER_SYMBOL     = "other_symbol"      # same name, different definition
    OUT_OF_SCOPE     = "out_of_scope"      # file outside the analyzed set
    UNPARSEABLE      = "unparseable"       # file failed to parse
    UNRESOLVED       = "unresolved"        # HARD ERROR: resolvable ref not rewritten

class RenameStatus(StrEnum):
    DRY_RUN = "dry_run"; APPLIED = "applied"
    REFUSED = "refused"; ABORTED = "aborted"; ROLLED_BACK = "rolled_back"

@dataclass(frozen=True, slots=True)
class SymbolTarget:
    module: str          # dotted defining module, e.g. "orchestrata_ultimate.knowledge_graph.config"
    name: str            # symbol name, e.g. "Neo4jConfig"
    new_name: str
    kind: str = "auto"   # auto|class|function|variable|method

@dataclass(frozen=True, slots=True)
class TextEdit:
    line: int; start_col: int; end_col: int; new_text: str   # 0-based, LSP-native

@dataclass(frozen=True, slots=True)
class FileEdit:
    path: Path; edits: tuple[TextEdit, ...]

@dataclass(frozen=True, slots=True)
class WorkspaceEditPlan:
    files: tuple[FileEdit, ...]
    @property
    def edit_count(self) -> int: ...
    @property
    def file_count(self) -> int: ...

@dataclass(frozen=True, slots=True)
class ResidueItem:
    path: Path; line: int; column: int
    text: str            # the source line, stripped
    reason: ResidueReason
    detail: str = ""

@dataclass(frozen=True, slots=True)
class RenameOutcome:
    target: str
    status: RenameStatus
    plan: WorkspaceEditPlan
    residue: tuple[ResidueItem, ...]
    errors: tuple[str, ...] = ()
    def to_dict(self) -> dict[str, Any]: ...   # machine-readable report

Rename plan schema (TOML; JSON accepted with identical keys)

version = 1
repo_root = "."

[[symbols]]
module   = "orchestrata_ultimate.knowledge_graph.config"
name     = "Neo4jConfig"
new_name = "KnowledgeGraphNeo4jConfig"

[[modules]]
old = "orchestrata_ultimate.orchestrator.redis_state"
new = "orchestrata_ultimate.orchestrator.state.redis_state"

[[strings]]
old = "scripts/old_name.py"
new = "scripts/new_name.py"
kind = "path"            # path | dotted | exact

keep_patterns = ["docs/**", "CHANGELOG.md", "**/*.pl"]

Functional Requirements

FR-001 — LSP transport

• REQ-109-001: Encode/decode LSP Content-Length framing over stdio.
• REQ-109-002: Correlate requests to responses by integer id, with per-request timeout.
• REQ-109-003: Answer server→client requests (workspace/configuration, client/registerCapability, window/workDoneProgress/create); an unanswered workspace/configuration deadlocks the handshake.
• REQ-109-004: Full lifecycle — initialize, initialized, shutdown, exit — and the server process is reaped on exit and on error.

FR-002 — Engine-owned workspace configuration

• REQ-109-005: Generate a temporary directory containing a pyrightconfig.json and use it as the LSP workspace root; the repo's own config is never read, written, or mutated.
• REQ-109-006: The generated config uses absolute include (default: src, tests, scripts), exclude, and extraPaths (the repo source root) pointing into the real repo.
• REQ-109-007: The generated config sets typeCheckingMode: "off" — the engine wants references, not diagnostics — and enables no basedpyright "based extras". (See SR-109-003.)
• REQ-109-008: The temporary config root is removed on shutdown, including on error.

FR-003 — Resolution preflight (anti-silent-under-rename guard)

• REQ-109-009: Before any rename, verify the langserver resolves the target module to the same on-disk file the engine opened; a mismatch aborts.
• REQ-109-010: If any analyzed file's import of the target module fails to resolve, abort with UNRESOLVED_IMPORTS before writing anything.
• REQ-109-011: If the candidate oracle finds cross-file candidates but the WorkspaceEdit contains edits in only the defining file, abort — this is the S2 signature.

FR-004 — Type-resolved symbol rename

• REQ-109-012: textDocument/didOpen the defining document before prepareRename; without it, rename is same-file-only (spike S1).
• REQ-109-013: prepareRename must return a range; a null result refuses the rename with a diagnostic rather than proceeding.
• REQ-109-014: Normalize both WorkspaceEdit shapes (changes map and documentChanges array) into one WorkspaceEditPlan.
• REQ-109-015: A SymbolTarget is resolved by defining module + name; the engine locates the definition site itself rather than accepting a raw position.
• REQ-109-016: A symbol with the same name defined in a different module is never edited.

FR-005 — Candidate oracle and residue classification

• REQ-109-017: An independent occurrence scan (textual, then AST-classified) over the analyzed file set produces the candidate set.
• REQ-109-018: Every candidate absent from the WorkspaceEditPlan is assigned exactly one ResidueReason.
• REQ-109-019: Classification distinguishes string literals, docstrings, comments, other-symbol bindings, out-of-scope files, and unparseable files.
• REQ-109-020: A candidate that is a resolvable code reference (an ast.Name/ast.Attribute/ImportFrom alias in a parsed, in-scope file) but is absent from the plan is classified UNRESOLVED and aborts the run.
• REQ-109-021: The residue report is machine-readable and carries path, line, column, source text, reason, and detail per item.

FR-006 — Atomic module move

• REQ-109-022: Move the file with git mv, preserving history; fall back to a filesystem move only when the path is untracked, and record which was used.
• REQ-109-023: Rewrite every absolute import of the old dotted path (import a.b, from a.b import x, from a import b) across the analyzed set.
• REQ-109-024: Rewrite string literals that provably denote the moved module — a whole-literal match of the old POSIX path or old dotted path.
• REQ-109-025: Refuse (do not guess) when a literal only partially matches, is built by concatenation/f-string, or matches more than one plan entry.
• REQ-109-026: The move is atomic — file move, import rewrites and string rewrites all land, or none do.

FR-007 — String-reference layer

• REQ-109-027: A string rename rewrites only whole-token literal matches of the declared old, under the declared kind (path/dotted/exact).
• REQ-109-028: keep_patterns (glob) exclude files from all string rewriting.
• REQ-109-029: Any occurrence matching old that is not a provable whole-token literal is emitted as residue, never rewritten.

FR-008 — Rename plan

• REQ-109-030: Load and validate a plan from TOML or JSON with identical keys.
• REQ-109-031: Each symbols entry requires module, name, new_name; a bare symbol name without a defining module is a validation error.
• REQ-109-032: Each modules entry requires old and new dotted paths.
• REQ-109-033: Each strings entry requires old, new, and kind.
• REQ-109-034: keep_patterns is honoured by every layer.
• REQ-109-035: The whole plan validates before any entry executes; validation errors name the offending entry index.

FR-009 — Atomic execution and rollback

• REQ-109-036: Dry run is the default; writing requires explicit opt-in.
• REQ-109-037: Execution snapshots every file it will touch, applies, verifies, and restores the snapshot on any failure (ROLLED_BACK).
• REQ-109-038: After applying, every touched .py file must parse (ast.parse); a parse failure triggers rollback.
• REQ-109-039: Distinct exit codes: 0 clean, 1 refused/aborted (residue or guard), 2 internal/transport error.

FR-010 — CLI

• REQ-109-040: scripts/rename_engine.py exposes symbol, module, plan, and report subcommands.
• REQ-109-041: Every mutating subcommand is dry-run unless --apply is passed, and prints the residue report regardless.

FR-011 — Volk tool

• REQ-109-042: A RefactorRenameTool conforms to the SPEC-072 BaseTool contract; ToolResult.output is the JSON RenameOutcome.
• REQ-109-043: Its metadata sets requires_approval=True and category ToolCategory.FILE.
• REQ-109-044: execute never raises; every failure becomes ToolResult(success=False, error=...).

FR-012 — Observability

• REQ-109-045: Every component composes an injected Tracer bound to a TraceConfig (core.tracing.tracer); no component prints or hard-codes a log level. This is separate from @traces requirement traceability.

Error Conditions

Security Requirements

• SR-109-001 (REQ-109-201): Every path is resolved and confined to the declared repo_root; traversal outside it is rejected before any I/O.
• SR-109-002 (REQ-109-202): The engine writes only inside repo_root and its own temporary config root; it never writes to the repo's pyrightconfig.json, pyproject.toml, or .git internals beyond git mv.
• SR-109-003 (REQ-109-203): The generated pyright config never enables basedpyright "based extras" and the engine never invokes the pyright console script, so scripts/pyright_ratchet.py (--max 283) and scripts/check_async_await.py are unaffected.
• SR-109-004 (REQ-109-204): No shell string interpolation — git mv and the langserver are spawned as argument vectors, never through a shell.
• SR-109-005 (REQ-109-205): Environment is read via os.environ.get with defaults; no bare os.environ[...].

API Contracts

# refactor/rename/lsp_client.py
class PyrightLangServer:
    def __init__(self, config: LspServerConfig, tracer: Tracer | None = None) -> None: ...
    def __enter__(self) -> PyrightLangServer: ...
    def __exit__(self, *exc: object) -> None: ...
    def open_document(self, path: Path) -> None: ...
    def prepare_rename(self, path: Path, line: int, column: int) -> dict | None: ...
    def rename(self, path: Path, line: int, column: int, new_name: str) -> WorkspaceEditPlan: ...
    def definition(self, path: Path, line: int, column: int) -> list[dict]: ...

# refactor/rename/workspace.py
class WorkspaceConfigBuilder:
    def build(self, repo_root: Path, include: Sequence[str], exclude: Sequence[str]) -> Path: ...
    def cleanup(self) -> None: ...

class ResolutionPreflight:
    def check(self, server: PyrightLangServer, target: SymbolTarget,
              candidates: Sequence[ResidueItem]) -> tuple[bool, tuple[str, ...]]: ...

# refactor/rename/symbol_rename.py
class SymbolRenamer:
    def __init__(self, server, preflight, tracer=None) -> None: ...
    def plan(self, target: SymbolTarget) -> WorkspaceEditPlan: ...

# refactor/rename/scanner.py
class CandidateScanner:
    def scan(self, name: str, roots: Sequence[Path]) -> tuple[ResidueItem, ...]: ...

# refactor/rename/residue.py
class ResidueClassifier:
    def classify(self, candidates, plan: WorkspaceEditPlan,
                 analyzed_roots: Sequence[Path]) -> tuple[ResidueItem, ...]: ...

# refactor/rename/applier.py
class EditApplier:
    def apply(self, plan: WorkspaceEditPlan) -> None: ...      # raises on verify failure
    def dry_run_diff(self, plan: WorkspaceEditPlan) -> str: ...

# refactor/rename/module_move.py
class ModuleMover:
    def plan(self, old: str, new: str) -> tuple[WorkspaceEditPlan, tuple[ResidueItem, ...]]: ...
    def execute(self, old: str, new: str, *, apply: bool = False) -> RenameOutcome: ...

# refactor/rename/plan.py / executor.py
def load_plan(path: Path) -> RenamePlan: ...
class PlanExecutor:
    def execute(self, plan: RenamePlan, *, apply: bool = False) -> tuple[RenameOutcome, ...]: ...

Dependencies

Rejected alternatives (recorded so they are not re-litigated):

• Pylance — proprietary; violates the open-source-only ruling.
• rope (LGPL) — retained as the fallback only if basedpyright had failed; the spike passed, so rope is not a dependency.
• jedi — no cross-file rename with the required precision.
• Extending serialize_codebase_ast.py rename — name-based matching is the defect.

Testing Requirements

Real tests only: every LSP test spawns a real basedpyright-langserver against a real fixture package on tmp_path. No mocks, no monkeypatch, no skips, function-scope fixtures, no xdist.

Definition of Done

• [ ] refactor/rename/ implements FR-001…FR-009 with @traces(reqs=[...]) citing REQ-109-*.
• [ ] scripts/rename_engine.py CLI (FR-010).
• [ ] RefactorRenameTool class exists and conforms to SPEC-072 (FR-011). Registry wiring in agents/tool_registry.py is a separate, later step — not in scope here.
• [ ] TST-109-001…016 pass against a real langserver via the batched runner (never xdist).
• [ ] scripts/pyright_ratchet.py --max 283 still passes unchanged (SR-109-003).
• [ ] basedpyright declared in [dependency-groups] dev via uv add --dev.

Revision History

Related

• RTM page for 109-rename-engine
• All specs
• RTM overview
• Gates