Orchestrata Ultimate › Plans › tools_useful_to_plan

Tools Useful to Plan

Source: plans/tools_useful_to_plan.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-04-11 · 795 lines.

Prose pending.

Plan text

Tools Useful to Plan

Reference guide for Claude and other agents working on the Orchestrata-Ultimate port. Every tool lives in scripts/ unless noted. All commands assume the project root as CWD.


1. serialize_codebase_ast.py (1451 LOC)

Purpose: Source of truth for what exists in both codebases. Serializes every module, class, method, signature, import, and inheritance chain to JSON. Performs semantic comparison between orchestrata and orchestrata-ultimate.

Key classes: CodebaseAST, ClassInfo, FunctionInfo, ParamInfo, ImportInfo, ModuleInfo, ClassVarInfo

Subcommands

serialize — Produce a full AST JSON for one codebase
python scripts/serialize_codebase_ast.py serialize \
    --source src/orchestrata_ultimate \
    --name orchestrata-ultimate \
    --output plans/ast-ultimate.json
python scripts/serialize_codebase_ast.py serialize \
    --source /home/mohamed/workspace/orchestrata/packages \
    --name orchestrata \
    --output plans/ast-orchestrata.json
deep-compare — Semantic comparison using class mapping
python scripts/serialize_codebase_ast.py deep-compare \
    plans/ast-ultimate.json \
    plans/ast-orchestrata.json \
    --mapping plans/class-mapping.json \
    --output plans/ast-deep-comparison.json

Produces the 336K JSON the execution plan was built from. For each class pair it reports readiness (stub, partial, mostly_complete, needs_creation, equal), method-level signature diffs, inheritance chains, and class variable diffs. Also reports per-package coverage, import translations, and dependency diffs.

compare — Shallow name-based comparison
python scripts/serialize_codebase_ast.py compare \
    plans/ast-ultimate.json \
    plans/ast-orchestrata.json \
    --output plans/ast-comparison.json

Name-only match without semantic pairing. Less useful than deep-compare.

When to use


2. diagram_generator.py (1585 LOC)

Purpose: Bidirectional Code ↔ Mermaid diagram generator. Uses the full CodebaseAST from serialize_codebase_ast.py (not a watered-down model). Generates class diagrams from source, generates Python skeleton code from diagrams, and verifies round-trip fidelity.

Key classes: CodeToDiagram, DiagramToCode, CodebaseAnalyzer, SpecUpdater

Subcommands

code2diagram — Generate Mermaid from source
python scripts/diagram_generator.py code2diagram \
    --source src/orchestrata_ultimate \
    --output-dir specs/001-kernel

Options:

diagram2code — Generate Python skeleton from Mermaid
python scripts/diagram_generator.py diagram2code \
    --diagram specs/001-kernel/kernel.mmd \
    --output-dir src/orchestrata_ultimate/core
verify — Round-trip verification (code → diagram → code)
python scripts/diagram_generator.py verify \
    --source src/orchestrata_ultimate \
    --output-dir /tmp/verify_output

Reports class count matches, method set matches, and any mismatches.

Legacy modes (for spec updating)
# Update a single spec with diagrams
python scripts/diagram_generator.py --spec specs/001-kernel/spec.md

# Update all specs
python scripts/diagram_generator.py --all

# Dry run (show what would change)
python scripts/diagram_generator.py --all --dry-run

# Validate existing diagrams only
python scripts/diagram_generator.py --validate

When to use


3. spec-review.py (1080 LOC)

Purpose: Reviews all 55+ spec files in specs/*/spec.md against the 20 standards defined in specs/spec-review/spec-standards.yaml. Reports missing sections, incomplete data models, missing API contracts, etc.

# Review all specs
python scripts/spec-review.py

# Review a specific spec
python scripts/spec-review.py specs/001-kernel/spec.md

Output includes total specs reviewed, pass/fail counts, issues by section, and per-spec issue counts with criticality levels.

When to use


4. generate_comprehensive_audit.py (752 LOC)

Purpose: Generates a compliance-audit.md for each spec, comparing the spec against the actual codebase using CodebaseAST + CodeToDiagram. Reports which spec requirements are implemented, partially implemented, or missing.

Note: This script must be run from the scripts/ directory (uses local imports).

cd scripts && python generate_comprehensive_audit.py

The script currently has a hardcoded source path that needs the codebase at scripts/src/orchestrata_ultimate. Adjust the path or symlink if needed.

When to use


5. code-review.py (1163 LOC)

Purpose: Unified code review tool that runs ruff, mypy, and AST-based style checks. Produces reports in text, JSON, or markdown.

# Review specific files
python scripts/code-review.py src/orchestrata_ultimate/core/orchestrator.py

# Review a directory
python scripts/code-review.py src/orchestrata_ultimate/core/ --format markdown -o review.md

# Skip specific checks
python scripts/code-review.py src/orchestrata_ultimate/ --skip-mypy --skip-ruff

# Set minimum severity
python scripts/code-review.py src/orchestrata_ultimate/ --min-severity WARNING

Options:

When to use


6. architecture_checker.py (461 LOC)

Purpose: Validates project architecture — enforces import rules, module boundaries, dependency direction, and layering constraints.

# Check entire project
python scripts/architecture_checker.py src/orchestrata_ultimate/

# Check specific paths with verbose output
python scripts/architecture_checker.py src/orchestrata_ultimate/core/ -v

# Output to file
python scripts/architecture_checker.py src/orchestrata_ultimate/ -o arch_report.md

Options:

When to use


7. auto_fix.py (241 LOC)

Purpose: Auto-fixer for common LLM-generated code defects. Runs a pipeline of fixers over Python files.

Available fixers:

Fixer What it does
relative_imports Rewrites relative imports (from .. → absolute)
line_length Splits lines exceeding 88 characters
import_order Reorders imports: stdlib, third-party, local (separated by blank lines)
redis_host Replaces hardcoded "localhost" in Redis constructors with os.environ.get()
missing_datetime Adds missing datetime import when datetime is used bare
# Fix all Python files in a directory
python scripts/auto_fix.py src/orchestrata_ultimate/core/

# Fix specific files with verbose output
python scripts/auto_fix.py src/orchestrata_ultimate/core/orchestrator.py -v

# List available fixers
python scripts/auto_fix.py --list-fixers

# Disable specific fixers
python scripts/auto_fix.py src/orchestrata_ultimate/ --disable redis_host --disable line_length

When to use


8. fix_relative_imports.py (116 LOC) / fix_relative_imports_ast.py (139 LOC)

Purpose: Two scripts that rewrite relative imports to absolute imports. fix_relative_imports_ast.py uses AST-based parsing (more reliable); fix_relative_imports.py uses regex.

# Fix a specific file
python scripts/fix_relative_imports_ast.py src/orchestrata_ultimate/core/traceability/__init__.py

# Fix all files in a directory (no args = current dir)
python scripts/fix_relative_imports_ast.py

Prefer fix_relative_imports_ast.py — it is more accurate. Both modify files in-place and remove backup files.

When to use


9. generate_pytest.py (917 LOC)

Purpose: Generates pytest test files from AST analysis of source code. Creates test stubs with proper imports, fixtures, and test function signatures based on the classes and methods found.

# Generate tests for a single module
python scripts/generate_pytest.py src/orchestrata_ultimate/core/orchestrator.py -o tests/

# Generate tests for a directory
python scripts/generate_pytest.py src/orchestrata_ultimate/core/ -o tests/

# Use custom template directory
python scripts/generate_pytest.py src/orchestrata_ultimate/core/ -o tests/ -t templates/

Options:

When to use


10. generate_module.py (258 LOC)

Purpose: Generates Python module scaffolds from templates. Supports three module types: task, service, agent.

# Generate an agent module
python scripts/generate_module.py agent qa_volk \
    --class-name QualityAssuranceAgent \
    --description "QA verification agent" \
    --output src/orchestrata_ultimate/agents/volks/

# Generate a service module
python scripts/generate_module.py service cache \
    --class-name CacheService \
    --output src/orchestrata_ultimate/core/

# Generate a task module
python scripts/generate_module.py task validation \
    --output src/orchestrata_ultimate/kernel/

Options:

When to use


11. generate_docs.py (383 LOC)

Purpose: Generates markdown or README documentation from source code using AST analysis.

# Generate markdown docs for a module
python scripts/generate_docs.py src/orchestrata_ultimate/core/orchestrator.py -o docs/core.md

# Generate README format for a directory
python scripts/generate_docs.py src/orchestrata_ultimate/core/ -f readme -o docs/core_README.md

# Use a custom template
python scripts/generate_docs.py src/orchestrata_ultimate/core/ -t custom_template.md

Options:

When to use


12. enhance_spec.py (462 LOC)

Purpose: Analyzes spec files and adds missing sections based on the standards defined in specs/spec-review/spec-standards.yaml. Inserts placeholder sections for required content.

# Preview what would be added to a spec (dry run, default)
python scripts/enhance_spec.py specs/001-kernel/spec.md

# Actually apply changes
python scripts/enhance_spec.py specs/001-kernel/spec.md --apply

# Process all specs
python scripts/enhance_spec.py --all --apply

# JSON output
python scripts/enhance_spec.py specs/001-kernel/spec.md --output json

Options:

When to use


13. generate_rtm.py (145 LOC) / generate_complete_rtm.py (189 LOC) / analyze_orchestrata_rtm.py (205 LOC)

Purpose: Requirements Traceability Matrix tools. They parse requirements from specs, find @traces annotations in source code, and map requirements to implementation.

generate_rtm.py

Basic RTM generation from source code trace annotations.

python scripts/generate_rtm.py

Outputs rtm/rtm.json and rtm/rtm.md.

generate_complete_rtm.py

Extended RTM that also includes test file mappings and integrates with orchestrata's RTM data.

python scripts/generate_complete_rtm.py

Outputs rtm/rtm.json, rtm/rtm.md, rtm/comprehensive_mapping.json, and rtm/integrated_rtm.json.

analyze_orchestrata_rtm.py

Analyzes the orchestrata source RTM entries and maps them to orchestrata-ultimate paths. Produces a coverage report.

python scripts/analyze_orchestrata_rtm.py

Outputs rtm/orchestrata_coverage_report.md and rtm/orchestrata_rtm_sample_transformed.json.

When to use


14. parse_requirements.py (162 LOC)

Purpose: Parses requirements from spec files and produces structured JSON. Used by RTM generators.

Note: Currently broken due to an import error (ModuleNotFoundError: No module named 'orchestrata_ultimate.core.traceability_utils'). This needs a fix in src/orchestrata_ultimate/core/traceability/__init__.py before it can run.

python scripts/parse_requirements.py

When to use


15. run_tests.sh (264 LOC) / run_tests_parallel.py (155 LOC) / run_prolog_tests.py (241 LOC) / run_plunit_tests.py (319 LOC)

Purpose: Test runners with different specializations.

run_tests.sh

Parallel per-module test runner. Each module gets a fresh Prolog engine to avoid pyswip's global singleton issues.

# Run all tests in parallel
./scripts/run_tests.sh

# Force sequential execution
./scripts/run_tests.sh --serial

# Use Python runner instead of shell
./scripts/run_tests.sh --python

# Pass extra pytest args
./scripts/run_tests.sh -v --tb=short

Note: Shell runner is deprecated in favor of the Python runner. Use --python flag or call run_tests_parallel.py directly.

run_tests_parallel.py

Python-based parallel test runner. Currently broken due to import error.

python scripts/run_tests_parallel.py
python scripts/run_tests_parallel.py --serial

run_prolog_tests.py

Runs Prolog test files per-file with crash recovery and JUnit XML output.

# Run all Prolog tests
python scripts/run_prolog_tests.py

# Run with coverage
python scripts/run_prolog_tests.py --cov

# Run with pytest marker filter
python scripts/run_prolog_tests.py --marker governance

# Specify test directories
python scripts/run_prolog_tests.py --dirs tests/knowledge tests/governance

# Output JUnit XML
python scripts/run_prolog_tests.py --junit-xml test-prolog.xml

run_plunit_tests.py

Runs SWI-Prolog plunit tests and emits JUnit XML.

# Run all plunit tests
python scripts/run_plunit_tests.py

# Filter by file pattern
python scripts/run_plunit_tests.py --file test_ethics.plt

# Custom plt directory
python scripts/run_plunit_tests.py --plt-dir prolog/tests/

# JUnit XML output
python scripts/run_plunit_tests.py --junit-xml junit.xml

When to use


16. check_test_infrastructure.sh (60 LOC) / setup_test_infrastructure.sh (671 LOC)

Purpose: Infrastructure health check and setup for running real (non-mock) tests against Redis, Neo4j, and SWI-Prolog.

check_test_infrastructure.sh

Verifies that test infrastructure is running:

bash scripts/check_test_infrastructure.sh

Checks: Redis (port 6380, DB 15), Neo4j (port 7688), SWI-Prolog, test directories.

setup_test_infrastructure.sh

Sets up the full test infrastructure (Docker containers, DBs, message queues):

bash scripts/setup_test_infrastructure.sh

Creates: Redis test instance (port 6380, DB 15), Neo4j test instance (port 7688), Prolog test server (port 8080), temp directories.

When to use


17. debug_mypy.py (34 LOC)

Purpose: Runs mypy on the project and parses the output into structured error reports.

python scripts/debug_mypy.py

Parses mypy errors and prints them in a more readable format. Currently shows errors in thread_pool.py (union-attr, no-any-return, var-annotated).

When to use


18. update_opencode_font.py (35 LOC)

Purpose: Updates the opencode font configuration to use MesloLGL Nerd Font Mono.

python scripts/update_opencode_font.py

Sets: fontSize=14, mono=MesloLGL Nerd Font Mono, font=ibm-plex-mono.

When to use


19. session_cli.py (94 LOC)

Purpose: CLI for session management — tracking context, achievements, and session state.

Note: Currently broken due to the same traceability_utils import error.

python scripts/session_cli.py status
python scripts/session_cli.py start
python scripts/session_cli.py achievement <name>

When to use


20. test_diagram.py (57 LOC) / test_diagram2.py (74 LOC)

Purpose: Legacy test scripts for the diagram generator. Both are currently broken due to importing old class names (ASTAnalyzer, MermaidGenerator) that no longer exist in the refactored diagram_generator.py.

Status: Broken — should be deleted or updated to use the current CodeToDiagram/DiagramToCode API.

When to use


Key Data Files

File Purpose
plans/ast-ultimate.json Full AST of orchestrata-ultimate (397K)
plans/ast-orchestrata.json Full AST of orchestrata source (8.0MB)
plans/ast-deep-comparison.json Semantic comparison with all diffs (336K)
plans/ast-comparison.json Shallow name-based comparison
plans/class-mapping.json Explicit class name mappings + regex transforms + module mappings + tier assignments
plans/execution-plan.md 238-step execution plan for the port
specs/spec-review/spec-standards.yaml 20 standards for spec compliance
rtm/rtm.json Requirements traceability matrix

Common Workflows

After completing a tier of the execution plan

# 1. Re-serialize the ultimate codebase
python scripts/serialize_codebase_ast.py serialize \
    --source src/orchestrata_ultimate \
    --name orchestrata-ultimate \
    --output plans/ast-ultimate.json

# 2. Deep-compare against orchestrata
python scripts/serialize_codebase_ast.py deep-compare \
    plans/ast-ultimate.json \
    plans/ast-orchestrata.json \
    --mapping plans/class-mapping.json \
    --output plans/ast-deep-comparison.json

# 3. Update spec diagrams
python scripts/diagram_generator.py --all

# 4. Review specs
python scripts/spec-review.py

# 5. Run tests
uv run pytest tests/ -n auto -x --timeout=120

After porting a single file

# 1. Fix imports
python scripts/auto_fix.py src/orchestrata_ultimate/<path>/

# 2. Code review
python scripts/code-review.py src/orchestrata_ultimate/<path>/

# 3. Architecture check
python scripts/architecture_checker.py src/orchestrata_ultimate/<path>/ -v

# 4. Generate tests
python scripts/generate_pytest.py src/orchestrata_ultimate/<path>/ -o tests/<path>/

# 5. Run relevant tests
uv run pytest tests/<path>/ -n auto -x --timeout=120

Checking overall porting progress

# Re-serialize and deep-compare (see above), then inspect the output:
python -c "
import json
d = json.load(open('plans/ast-deep-comparison.json'))
for tier, info in d.get('tier_analysis', {}).items():
    print(f'{tier}: {info.get(\"coverage_pct\", 0):.1f}% coverage')
"

21. sync_ast_to_neo4j.py (306 LOC) — CLI wrapper

Purpose: Syncs the serialized AST JSON to Neo4j via Neo4jSyncer engine (src/orchestrata_ultimate/knowledge_graph/sync.py, 1,986 LOC). 12 node types, 18 relationship types.

python scripts/sync_ast_to_neo4j.py schema                              # Create constraints + indexes
python scripts/sync_ast_to_neo4j.py full --ast plans/ast-ultimate.json --project orchestrata-ultimate  # Full sync
python scripts/sync_ast_to_neo4j.py stats                               # Node/edge counts
python scripts/sync_ast_to_neo4j.py verify --project orchestrata-ultimate  # 8 verification checks
python scripts/sync_ast_to_neo4j.py incremental --ast plans/ast-ultimate.json --project orchestrata-ultimate --modules agents.base_agent  # Re-sync specific modules
python scripts/sync_ast_to_neo4j.py map --mapping plans/class-mapping.json  # Cross-project MAPS_TO edges

Connection via env vars (NEO4J_URI, NEO4J_USERNAME, NEO4J_PASSWORD) or CLI flags (--uri, --user, --password).


22. perplexity_search.py (89 LOC) — Perplexity API Integration

Purpose: Command-line interface to query the Perplexity AI API for web search and information retrieval.

# Basic usage
python scripts/perplexity_search.py "What is the latest Python version?"

# Specify model
python scripts/perplexity_search.py --model pplx-70b-online "Advanced query"

# Output raw JSON response
python scripts/perplexity_search.py --raw "Debug query"

# Use specific API key
python scripts/perplexity_search.py --api-key your-key-here "Query"

Environment Variables:

Available Models:

When to use


23. scripts/orchestrator/ (7 files, 2,110 LOC) — Agent Dispatch Pipeline

Purpose: Automates the sandboxed agent workflow. Agents work in isolated directories with only designated files writable. Orchestrator handles context injection, test execution, verification, and git operations.

Config-driven via YAML. See scripts/orchestrator/README.md for config format and docs/ORCHESTRATOR.md for architecture.

python scripts/orchestrator/preflight.py --config agent_config.yaml         # 8 env checks
python scripts/orchestrator/create_sandbox.py --config agent_config.yaml    # Build sandbox + context
python scripts/orchestrator/generate_prompt.py --config agent_config.yaml --output /tmp/prompt.md  # Render prompt
python scripts/orchestrator/verify_agent.py --config agent_config.yaml      # Copy→test→diff
python scripts/orchestrator/validate_audit.py --config agent_config.yaml    # Validate AUDIT.md
python scripts/orchestrator/merge_agent.py --config agent_config.yaml       # Branch→commit→merge (no push)

Pipeline: preflight → create_sandbox → generate_prompt → [dispatch agent] → verify_agent → validate_audit → [human review] → merge_agent


Known Issues

  1. Import error in src/orchestrata_ultimate/core/traceability/__init__.py — references orchestrata_ultimate.core.traceability_utils which doesn't exist. This breaks parse_requirements.py, run_tests_parallel.py, session_cli.py, and generate_comprehensive_audit.py.

  2. test_diagram.py and test_diagram2.py — import old class names that no longer exist. Should be deleted or updated.

  3. generate_comprehensive_audit.py — must be run from scripts/ directory and has a hardcoded source path. Needs path adjustment.

  4. run_tests.sh — marked as deprecated in favor of run_tests_parallel.py (or plain uv run pytest).

  5. run_tests_parallel.py — broken due to the traceability import error.