Orchestrata Ultimate › Plans › scripts_fix › fix-plan

Scripts Fix Plan

Source: plans/scripts_fix/fix-plan.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-04-14 · 608 lines.

Prose pending.

Plan text

Scripts Fix Plan

18 issues across 15 scripts. All deterministic fixes, no architectural changes. Generated from code review of all 35 scripts (~18,383 LOC).

How to Use This Plan

1. Execute one step at a time.
2. Run the verification command after each step.
3. Commit only after all steps pass (gate check at the end).

Phase 1: Critical Bugs (Steps 1–6)

Step 1: Fix LineLengthFixer undefined variable i

File: scripts/auto_fix.py:93 Issue: Loop uses for line in lines but body references lines[i] — i is never defined. Severity: CRITICAL — NameError on every invocation.

Actions:

1. Open scripts/auto_fix.py
2. Replace the broken loop (lines 86–99) with:

for i, line in enumerate(lines):
    if len(line) > 88 and not line.strip().startswith("#"):
        if "," in line:
            split_idx = line.rfind(",", 0, 88)
            if split_idx > 0:
                lines[i] = (
                    line[: split_idx + 1]
                    + "\n    "
                    + line[split_idx + 1 :].lstrip()
                )
                modified = True

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
from scripts.auto_fix import LineLengthFixer
f = LineLengthFixer()
result = f._fix_content('x = some_very_long_function_call(arg1, arg2, arg3, arg4, arg5, arg6)', __import__('pathlib').Path('test.py'))
print('OK' if result else 'OK (no change)')
"

Expect: OK (no NameError)

Step 2: Fix RedisHostFixer broken regex replacement

File: scripts/auto_fix.py:119-125 Issue: Replacement string r"redis.\1Redis(host=os.environ.get('REDIS_HOST', '\2')" has wrong backreferences, missing closing paren, doesn't preserve original call structure.

Actions:

1. Open scripts/auto_fix.py
2. Replace the _fix_content method of RedisHostFixer (lines 118-125):

def _fix_content(self, content: str, file_path: Path) -> str:
    # Pattern 1: redis.Redis(host='localhost'...)
    new_content = re.sub(
        r"(redis\.(?:asyncio\.)?Redis\()host\s*=\s*['\"](?:localhost|127\.0\.0\.1)['\"]",
        r"\1host=os.environ.get('REDIS_HOST', 'localhost')",
        content,
    )
    # Add os import if needed
    if new_content != content and "import os" not in content:
        new_content = "import os\n" + new_content
    return new_content

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
from scripts.auto_fix import RedisHostFixer
f = RedisHostFixer()
result = f._fix_content(\"client = redis.Redis(host='localhost', port=6379)\", __import__('pathlib').Path('test.py'))
assert 'os.environ' in result, result
print('OK')
"

Expect: OK

Step 3: Fix Python 2 except syntax in generate_pytest.py (line 449)

File: scripts/generate_pytest.py:449 Issue: except AttributeError, NotImplementedError, RecursionError: is Python 2 syntax. Crashes on Python 3.14.

Actions:

1. Open scripts/generate_pytest.py
2. Line 449: replace except AttributeError, NotImplementedError, RecursionError: with except (AttributeError, NotImplementedError, RecursionError):

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
import ast, sys
with open('scripts/generate_pytest.py') as f:
    source = f.read()
ast.parse(source)
print('Syntax OK')
"

Expect: Syntax OK

Step 4: Fix Python 2 except syntax in generate_pytest.py (line 501)

File: scripts/generate_pytest.py:501 Issue: except AttributeError, NotImplementedError: — same Python 2 syntax.

Actions:

1. Open scripts/generate_pytest.py
2. Line 501: replace except AttributeError, NotImplementedError: with except (AttributeError, NotImplementedError):

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
import ast
with open('scripts/generate_pytest.py') as f:
    ast.parse(f.read())
print('Syntax OK')
"

Expect: Syntax OK

Step 5: Fix wrong filename in analyze_method_coverage.py

File: scripts/analyze_method_coverage.py:10 Issue: Hardcoded plans/ast-deep-comparison-fresh.json — actual file is plans/ast-deep-comparison.json. Script always fails with FileNotFoundError.

Actions:

1. Open scripts/analyze_method_coverage.py
2. Line 10: replace "plans/ast-deep-comparison-fresh.json" with "plans/ast-deep-comparison.json"

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python scripts/analyze_method_coverage.py 2>&1 | head -5

Expect: Prints # Method-Level Coverage Analysis header (not File not found)

Step 6: Fix wrong filename in analyze_porting_status.py

File: scripts/analyze_porting_status.py:10 Issue: Same wrong filename as Step 5.

Actions:

1. Open scripts/analyze_porting_status.py
2. Line 10: replace "plans/ast-deep-comparison-fresh.json" with "plans/ast-deep-comparison.json"

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python scripts/analyze_porting_status.py 2>&1 | head -5

Expect: Prints # Porting Status Report header (not File not found)

Phase 2: Non-Portable Paths (Steps 7–8)

Step 7: Fix hardcoded absolute paths in spec-review.py

File: scripts/spec-review.py:21-23 Issue: Hardcoded /home/mohamed/workspace/orchestrata-ultimate/specs — breaks on any other machine.

Actions:

1. Open scripts/spec-review.py
2. Replace lines 21-23:

# Configuration
PROJECT_ROOT = Path(__file__).resolve().parent.parent
SPECS_DIR = PROJECT_ROOT / "specs"
REVIEWS_DIR = PROJECT_ROOT / "reviews"
SPEC_STANDARD = PROJECT_ROOT / "SPEC_STANDARD.md"

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
from pathlib import Path
import sys
sys.path.insert(0, 'scripts')
import importlib.util
spec = importlib.util.spec_from_file_location('spec_review', 'scripts/spec-review.py')
mod = importlib.util.module_from_spec(spec)
# Just check it loads without error
spec.loader.exec_module(mod)
print(f'SPECS_DIR = {mod.SPECS_DIR}')
assert mod.SPECS_DIR.exists(), f'SPECS_DIR does not exist: {mod.SPECS_DIR}'
print('OK')
"

Expect: OK with correct path

Step 8: Fix broken import path in enhance_spec.py

File: scripts/enhance_spec.py:20-21 Issue: sys.path.insert(0, os.path.join(..., "specs", "spec-review")) — directory name spec-review has a hyphen, not a valid Python package name.

Actions:

1. Open scripts/enhance_spec.py
2. Replace lines 20-53. The review_spec module doesn't exist as a valid import. Replace the import block with a graceful fallback:

# Try to import review_spec — may not exist
try:
    from specs.spec_review.review_spec import (  # noqa: F401
        STRUCTURAL_CHECKS,
        SpecReport,
        Standard,
        StructuralCheck,
        check_structure,
        extract_metadata,
        load_standards,
        review_spec,
    )
    REVIEW_SPEC_AVAILABLE = True
except (ImportError, ModuleNotFoundError):
    REVIEW_SPEC_AVAILABLE = False
    STRUCTURAL_CHECKS = []
    SpecReport = None
    Standard = None
    StructuralCheck = None
    check_structure = None
    extract_metadata = None
    load_standards = None
    review_spec = None

3. In enhance_spec() function (line 312), add a guard at the top:

def enhance_spec(spec_path: str, dry_run: bool = True) -> dict:
    """Enhance a spec file by adding missing sections."""
    if not REVIEW_SPEC_AVAILABLE:
        return {
            "spec": spec_path,
            "added": [],
            "skipped": "review_spec module not available",
        }

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
from scripts.enhance_spec import enhance_spec
result = enhance_spec('specs/001-ultimate-architecture/spec.md')
print(f'Result: {result[\"skipped\"] if \"skipped\" in result else \"added \" + str(len(result[\"added\"])) + \" sections\"}')
print('OK')
"

Expect: OK (no crash, graceful skip if module unavailable)

Phase 3: Duplicate Cleanup (Steps 9–10)

Step 9: Delete fix_relative_imports.py (regex version, keep AST version)

File: scripts/fix_relative_imports.py Issue: Duplicate of fix_relative_imports_ast.py. The regex version is less accurate. The AST version is the correct one.

Actions:

1. Delete scripts/fix_relative_imports.py

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && test ! -f scripts/fix_relative_imports.py && echo "OK: deleted" || echo "FAIL: still exists"

Expect: OK: deleted

Step 10: Merge generate_rtm.py and generate_complete_rtm.py

File: scripts/generate_rtm.py and scripts/generate_complete_rtm.py Issue: ~90% identical code. generate_complete_rtm.py is the superset (includes integrate_with_orchestrata_rtm()).

Actions:

1. Delete scripts/generate_rtm.py (keep generate_complete_rtm.py as the canonical version)

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && test ! -f scripts/generate_rtm.py && echo "OK: deleted" || echo "FAIL: still exists"

Expect: OK: deleted

Phase 4: Stale References (Steps 11–18)

Step 11: Update docstring in serialize_codebase_ast.py

File: scripts/serialize_codebase_ast.py:32-35 Issue: Docstring claims output sizes 397K/336K — actual are 14M/1.1M.

Actions:

1. Open scripts/serialize_codebase_ast.py
2. Replace lines 32-35:

    plans/ast-ultimate.json        — 14M, full AST of ultimate
    plans/ast-orchestrata.json     — 8.0MB, 456 modules, 947 classes, 121,692 LOC
    plans/ast-comparison.json      — 24K, shallow comparison
    plans/ast-deep-comparison.json — 1.1M, 34 pairs, 240 translations

Verify: No verification needed (docstring only).

Step 12: Fix architecture_checker.py layer directory references

File: scripts/architecture_checker.py:66-72 Issue: References core/ports, adapters, infrastructure, application directories that don't exist in current layout. Only domain exists under core/.

Actions:

1. Open scripts/architecture_checker.py
2. Replace the LAYERS dict (lines 66-72) to match actual project structure:

LAYERS = {
    "domain": ["src/orchestrata_ultimate/core/domain"],
    "protocols": ["src/orchestrata_ultimate/core/protocols"],
    "agents": ["src/orchestrata_ultimate/agents"],
    "core": ["src/orchestrata_ultimate/core"],
    "providers": ["src/orchestrata_ultimate/providers"],
    "knowledge_graph": ["src/orchestrata_ultimate/knowledge_graph"],
    "governance": ["src/orchestrata_ultimate/governance"],
}

3. Update DEPENDENCY_RULES (lines 75-82):

DEPENDENCY_RULES = {
    "domain": [],  # Domain cannot import any other layer
    "protocols": ["domain"],
    "agents": ["domain", "protocols", "core"],
    "core": ["domain", "protocols"],
    "providers": ["core", "protocols"],
    "knowledge_graph": ["core"],
    "governance": ["core", "knowledge_graph"],
}

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
from scripts.architecture_checker import LAYERS, DEPENDENCY_RULES
from pathlib import Path
for layer, dirs in LAYERS.items():
    for d in dirs:
        p = Path(d)
        if not p.exists():
            print(f'WARN: {d} does not exist (layer: {layer})')
print('OK')
"

Expect: OK (WARNs for non-existent dirs are acceptable)

Step 13: Remove duplicate abstractmethod in diagram_generator.py

File: scripts/diagram_generator.py:375 Issue: abstractmethod appears twice in UML_METHOD_DECORATORS set.

Actions:

1. Open scripts/diagram_generator.py
2. Remove the duplicate line 375 (second abstractmethod entry)

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
from scripts.diagram_generator import CodeToDiagram
c2d = CodeToDiagram.__new__(CodeToDiagram)
assert len(c2d.UML_METHOD_DECORATORS) == len(set(c2d.UML_METHOD_DECORATORS)), 'Duplicates found'
print('OK')
"

Expect: OK

Step 14: Remove unnecessary f-string in diagram_generator.py

File: scripts/diagram_generator.py:115 Issue: lines = [f"classDiagram"] — no interpolation, unnecessary f-string.

Actions:

1. Open scripts/diagram_generator.py
2. Line 115: replace lines = [f"classDiagram"] with lines = ["classDiagram"]

Verify: No verification needed (cosmetic).

Step 15: Fix run_tests_parallel.py unconventional import

File: scripts/run_tests_parallel.py:17-19 Issue: from src.orchestrata_ultimate.core.threading.thread_pool import ParallelTestExecutor — uses src. prefix which conflicts with project's absolute-import-only rule.

Actions:

1. Open scripts/run_tests_parallel.py
2. Replace line 17-19:

from orchestrata_ultimate.core.threading.thread_pool import (
    ParallelTestExecutor,
)

3. Remove the sys.path.insert(0, str(project_root)) on line 14 — not needed with proper absolute imports.

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
from orchestrata_ultimate.core.threading.thread_pool import ParallelTestExecutor
print('OK')
"

Expect: OK

Step 16: Fix generate_comprehensive_audit.py fragile import

File: scripts/generate_comprehensive_audit.py:18 Issue: sys.path.insert(0, str(Path(__file__).parent)) then bare imports from sibling modules — fragile and breaks if run from different working directory.

Actions:

1. Open scripts/generate_comprehensive_audit.py
2. Replace lines 18-34 with:

import sys
from pathlib import Path

# Allow imports from sibling scripts
_SCRIPTS_DIR = Path(__file__).parent
if str(_SCRIPTS_DIR) not in sys.path:
    sys.path.insert(0, str(_SCRIPTS_DIR))

from diagram_generator import (
    CodeToDiagram,
    DiagramToCode,
    ParsedClass,
    RelationshipKind,
    validate_mermaid,
)
from serialize_codebase_ast import (
    ClassInfo,
    ClassVarInfo,
    CodebaseAnalyzer,
    CodebaseAST,
    FunctionInfo,
    ParamInfo,
)

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
import sys
sys.path.insert(0, 'scripts')
from generate_comprehensive_audit import compare_designed_vs_implemented
print('OK')
"

Expect: OK

Step 17: Remove emoji from code-review.py output

File: scripts/code-review.py:975-983 Issue: Emoji characters (❌ ⚠️ ℹ️ 💡) in output break in CI/pipe-captured output.

Actions:

1. Open scripts/code-review.py
2. Line 975: replace the emoji dict with plain text:

severity_marker = {"ERROR": "[ERROR]", "WARNING": "[WARN]", "INFO": "[INFO]"}.get(
    issue.severity.value, "•"
)

3. Line 983: remove the 💡 prefix:

print(f"    Suggestion: {issue.suggestion}", file=output)

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
content = open('scripts/code-review.py').read()
assert '❌' not in content, 'Emoji still present'
assert '⚠️' not in content, 'Emoji still present'
assert 'ℹ️' not in content, 'Emoji still present'
assert '💡' not in content, 'Emoji still present'
print('OK')
"

Expect: OK

Step 18: Fix impact_analysis.py silent dotenv failure

File: scripts/impact_analysis.py:34-39 Issue: load_dotenv() silently swallowed on ImportError — .env may not load, causing auth failures.

Actions:

1. Open scripts/impact_analysis.py
2. Replace lines 34-39:

try:
    from dotenv import load_dotenv
    load_dotenv()
except ImportError:
    import warnings
    warnings.warn(
        "python-dotenv not installed. Set NEO4J_URI, NEO4J_USERNAME, "
        "NEO4J_PASSWORD environment variables manually.",
        UserWarning,
    )

Verify:

cd /home/mohamed/workspace/orchestrata-ultimate && python -c "
import warnings
warnings.filterwarnings('error')
try:
    exec(open('scripts/impact_analysis.py').read().split('def main')[0])
except UserWarning as e:
    print(f'WARN (expected if dotenv missing): {e}')
    print('OK')
except Exception as e:
    print(f'Unexpected: {e}')
else:
    print('OK (dotenv loaded)')
"

Expect: OK (either dotenv loaded or warning issued)

Phase 5: Gate Check

After all steps above are complete:

# 1. Syntax check all scripts
cd /home/mohamed/workspace/orchestrata-ultimate && python -m py_compile scripts/auto_fix.py && echo "auto_fix OK"
cd /home/mohamed/workspace/orchestrata-ultimate && python -m py_compile scripts/generate_pytest.py && echo "generate_pytest OK"
cd /home/mohamed/workspace/orchestrata-ultimate && python -m py_compile scripts/spec-review.py && echo "spec-review OK"
cd /home/mohamed/workspace/orchestrata-ultimate && python -m py_compile scripts/enhance_spec.py && echo "enhance_spec OK"
cd /home/mohamed/workspace/orchestrata-ultimate && python -m py_compile scripts/architecture_checker.py && echo "architecture_checker OK"
cd /home/mohamed/workspace/orchestrata-ultimate && python -m py_compile scripts/code-review.py && echo "code-review OK"
cd /home/mohamed/workspace/orchestrata-ultimate && python -m py_compile scripts/impact_analysis.py && echo "impact_analysis OK"
cd /home/mohamed/workspace/orchestrata-ultimate && python -m py_compile scripts/diagram_generator.py && echo "diagram_generator OK"
cd /home/mohamed/workspace/orchestrata-ultimate && python -m py_compile scripts/run_tests_parallel.py && echo "run_tests_parallel OK"
cd /home/mohamed/workspace/orchestrata-ultimate && python -m py_compile scripts/generate_comprehensive_audit.py && echo "generate_comprehensive_audit OK"

# 2. Verify deleted files are gone
test ! -f scripts/fix_relative_imports.py && echo "fix_relative_imports.py deleted OK"
test ! -f scripts/generate_rtm.py && echo "generate_rtm.py deleted OK"

# 3. Run ruff on all scripts
uv run ruff check scripts/ --fix

# 4. Run black on all scripts
uv run black scripts/

Expected: All compile, no deleted files remain, ruff and black pass clean.

Summary

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks