Orchestrata Ultimate › Plans › fix_test_template_issues

Plan: Fix Test Template Issues

Source: plans/fix_test_template_issues.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-04-11 · 94 lines.

Prose pending.

Plan text

Plan: Fix Test Template Issues

Issues Identified from Code Review

Based on code review of 24 generated test files (5685 total issues):

1. Import Issues (Highest Priority)

• F401: inspect and pytest imported but unused in each function
• I001: Import block is un-sorted or un-formatted
• F811: Redefinition of unused imports

Root Cause: Template imports inspect and pytest inside each test function instead of at module level.

Solution: Move import inspect and import pytest to module-level imports in template.

2. Type Annotation Issues

• MYPY_ERROR: Function is missing a return type annotation
• MISSING_RETURN_TYPE: Functions missing return type annotation

Solution: Add -> None return type to all test functions in template.

3. Style Issues

• LINE_TOO_LONG: Lines > 88 characters
• TRAILING_WHITESPACE: Whitespace at end of lines
• W293: Blank line contains whitespace

Solution: Fix template formatting to avoid long lines and trailing whitespace.

4. Naming Issues

• FUNCTION_NAMING: Function names like test_UltimateBaseError___init___interface flagged as non-snake_case

Note: This is a false positive - pytest test names with double underscores are valid. May need to adjust linter configuration.

5. Test Quality Issues

• TEST_PASS_STATEMENT: Tests contain pass statements (empty tests)
• MYPY_ERROR: Import-not-found errors for generated imports

Solution: Remove empty test generation for methods without parameters. Fix import paths.

Template Fixes Required

1. Interface Test Template (interface_test.jinja2)

• Move imports to module level
• Add -> None return type
• Fix line wrapping for long assertion messages
• Remove trailing whitespace

2. Validation Test Template (validation_test.jinja2)

• Move imports to module level
• Add -> None return type
• Fix get_test_value macro for None type handling
• Improve parameter validation logic for complex types
• Add proper type guards for union types

3. Test File Structure

• Module-level imports at top
• Proper function signatures with type hints
• Line length management
• No trailing whitespace

Implementation Steps

1. Update interface_test.jinja2:

   • Remove import inspect from function body
   • Add -> None to function definition
   • Wrap long lines using Jinja line continuation

2. Update validation_test.jinja2:

   • Remove import pytest from function body
   • Add -> None to function definitions
   • Fix get_test_value macro type checking
   • Add proper error handling for optional/union types

3. Update test file generation in generate_pytest.py:

   • Ensure imports are only added once at module level
   • Add proper module docstring
   • Configure mypy to ignore generated files

4. Run validation:

   • Regenerate tests
   • Run code review to verify issues reduced
   • Run pytest to ensure tests pass

Success Criteria

• Code review issues reduced by 90%+
• All generated tests pass mypy type checking
• No syntax errors in generated files
• Tests execute successfully with pytest

Timeline

• Immediate: Fix import and type annotation issues (1-2 hours)
• Short-term: Fix style and formatting issues (1 hour)
• Long-term: Improve template for complex type handling (future iteration)

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks