Knowledge Wiki Orchestrata Ultimate

Orchestrata Ultimate › Tests › Reference › refactor › ctok

Tests reference — tests/refactor/ctok

tests/refactor/ctok — 6 test modules, 83 test functions. Source tree: tests/refactor/ctok at commit 8a2cb42ad. Generated from the tests AST snapshot by scripts/gen-data-pages.py.

Prose pending.

Modules

Module Tests Covers
test_edit_buffer.py 21 1
test_lossless_module.py 9 2
test_node_index.py 16 3
test_token_stream.py 17 2
test_tokenizer.py 20 2

Module details

tests/refactor/ctok/test_edit_buffer.py

source · 21 tests · 210 lines

Tests for orchestrata_ultimate.refactor.ctok.edit_buffer.

Covers: refactor.ctok.edit_buffer

Kind Name Line Description
function test_add_rejects_negative_start 30 A negative start offset is never valid.
function test_add_rejects_start_after_end 37 start > end can never describe a real byte range.
function test_add_accepts_zero_width_span 44 start == end is a valid (pure-insertion) edit.
function test_apply_single_edit_splices_exactly 51 A single edit replaces exactly its span, nothing else.
function test_apply_out_of_bounds_raises 58 An edit whose end exceeds the source length is rejected.
function test_apply_overlapping_edits_raises 66 Two edits whose spans overlap are rejected -- ambiguous splice order.
function test_apply_multiple_non_overlapping_edits_all_land 75 Several non-overlapping edits all apply, added in any order.
function test_apply_untouched_bytes_are_byte_identical 83 A rename splice changes ONLY the target span; everything else is exact.
function test_apply_is_idempotent_pure_function 96 Calling apply() twice on the SAME source yields the SAME bytes.
function test_stringify_ast_ignores_source_position 111 Same AST shape, different whitespace/position -> identical dump.
function test_stringify_ast_differs_on_real_semantic_change 118 A genuinely different literal value produces a different dump.
function test_stringify_ast_raises_for_invalid_syntax 123 Unparsable source raises EditSafetyError, not a bare SyntaxError.
function test_verify_edit_accepts_formatting_only_change 134 Whitespace-only edits pass the equivalence gate.
function test_verify_edit_rejects_semantic_change_when_equivalence_claimed 139 A literal-value change claimed as formatting-only is rejected.
function test_verify_edit_accepts_valid_rename 145 Renaming ONLY a FunctionDef's name passes the rename gate.
function test_verify_edit_accepts_valid_name_rename 154 Renaming a Name (variable) identifier passes the rename gate.
function test_verify_edit_rejects_rename_that_also_changes_semantics 163 A rename that ALSO changes a literal is rejected -- not just-the-name.
function test_verify_edit_rejects_corrupting_splice_invalid_syntax 173 A splice that produces unparsable source is rejected as unsafe.
function test_verify_edit_rejects_when_before_itself_is_invalid 183 If 'before' fails to parse, verify_edit raises before comparing.
function test_verify_edit_idempotence_on_a_no_op_edit 189 Comparing identical before/after bytes is always safe, both modes.
function test_stringify_ast_skips_a_field_missing_from_a_hand_built_node 196 A manually-built ast node missing a declared field is skipped, not fatal.

tests/refactor/ctok/test_lossless_module.py

source · 9 tests · 141 lines

Tests for orchestrata_ultimate.refactor.ctok.lossless_module.LosslessModule.

Covers: refactor.ctok.edit_buffer, refactor.ctok.lossless_module

Kind Name Line Description
function test_parse_is_byte_identical_for_a_simple_module 27 The smallest possible sanity check: parse -> code round-trips.
function test_parse_exposes_tree_and_index 34 parse() wires up .tree (an ast.Module) and .index (a NodeIndex).
function test_parse_byte_identical_over_frozen_corpus 41 Every frozen codemod corpus fixture round-trips byte-identically.
function test_parse_byte_identical_whole_tree_sweep 54 Whole-tree sweep (P1 gate pattern) -- 0 mismatches, 0 parse errors.
function test_rename_returns_new_module_and_leaves_original_untouched 87 rename() never mutates self -- immutable edits (REQ-110-124).
function test_rename_changes_only_the_target_span 101 Bytes before and after the renamed span are byte-identical.
function test_replace_span_reparses_and_reindexes 114 replace_span() produces a module whose tree reflects the new bytes.
function test_replace_span_producing_invalid_syntax_raises 123 A splice that corrupts syntax fails loudly at apply/reparse time.
function test_apply_accepts_a_multi_edit_buffer 133 apply() splices every accumulated edit in one pass, then reparses.

tests/refactor/ctok/test_node_index.py

source · 16 tests · 226 lines

Tests for orchestrata_ultimate.refactor.ctok.node_index.NodeIndex.

Covers: refactor.ctok.lossless_module, refactor.ctok.node_index, refactor.ctok.tokenizer

Kind Name Line Description
function test_token_for_name_class_def_returns_class_identifier 43 ClassDef.name's token is the identifier after the class keyword.
function test_token_for_name_async_function_def_skips_both_keywords 52 async def bar must skip BOTH leading keyword tokens, not just one.
function test_token_for_name_function_def_returns_function_identifier 62 A plain (sync) FunctionDef's token is the name after def.
function test_token_for_name_name_node_returns_identifier_token 74 A bare ast.Name resolves directly to its own NAME token.
function test_token_for_name_attribute_returns_trailing_attr_not_value 87 Attribute rename target is .attr, never the value expression.
function test_span_for_slices_back_to_the_full_node_text 99 span_for's byte range, sliced from source, IS the node's own text.
function test_span_for_kind_is_the_first_covering_token 115 span_for's .kind tags the span with its FIRST token's kind.
function test_span_for_raises_for_node_without_location 132 A node with no lineno (e.g. a bare ctx node) cannot be spanned.
function test_span_for_raises_when_start_does_not_align_with_a_token 139 A fabricated node whose start lands inside a whitespace gap fails.
function test_token_for_name_raises_for_unsupported_node_type 151 token_for_name only supports Name/Attribute/def-name nodes.
function test_token_for_name_name_raises_when_unaligned 158 A fabricated Name pointing into a whitespace gap has no aligned token.
function test_token_for_name_attribute_raises_when_end_offset_unmatched 170 A fabricated Attribute whose end offset matches no token end fails.
function test_token_for_name_attribute_raises_when_resolved_token_is_not_name 184 A fabricated Attribute whose end offset lands on an OP token fails.
function test_token_for_name_class_def_raises_when_keyword_start_unaligned 198 A fabricated ClassDef whose keyword start is unaligned fails.
function test_token_for_name_class_def_raises_when_no_identifier_follows 210 A fabricated ClassDef over a keyword with no following identifier fails.
function test_build_does_not_require_tree_to_match_stream_tokens 222 NodeIndex.build's tree argument is accepted per the contract shape.

tests/refactor/ctok/test_token_stream.py

source · 17 tests · 185 lines

Tests for orchestrata_ultimate.refactor.ctok.token_stream.TokenStream.

Covers: refactor.ctok.token_stream, refactor.ctok.tokenizer

Kind Name Line Description
function test_reconstruct_empty_stream_returns_full_source_as_gap 19 No tokens at all: the whole source is one trailing gap.
function test_reconstruct_no_gaps_contiguous_tokens 26 Back-to-back tokens with zero inter-token gap reconstruct exactly.
function test_reconstruct_leading_gap_before_first_token 37 Bytes before the first token's start are gap-filled from source.
function test_reconstruct_inter_token_gap 45 Whitespace between two tokens (never itself a token) is gap-filled.
function test_reconstruct_trailing_gap_after_last_token 56 Bytes after the last token's end are gap-filled to end of source.
function test_reconstruct_zero_width_tokens_contribute_nothing 64 A zero-width token (start == end) adds no bytes but advances nothing.
function test_reconstruct_overlapping_or_backward_tokens_never_duplicate_bytes 76 The watermark (max(pos, end)) prevents double-emission on overlap.
function test_slice_returns_exact_byte_range 89 TokenStream.slice(a, b) is source[a:b], nothing else.
function test_token_stream_is_frozen_dataclass 98 TokenStream instances are immutable (frozen, slots).
function test_roundtrip_representative_module 110 A representative multi-construct module round-trips byte-identically.
function test_roundtrip_numeric_literals 131 Underscored/hex/oct/bin/complex numeric literals round-trip exactly.
function test_roundtrip_implicit_string_concatenation 146 Adjacent string literals (implicit concatenation) round-trip exactly.
function test_roundtrip_type_param_brackets_pep695 153 PEP 695 generic type-param brackets round-trip exactly.
function test_roundtrip_triple_quoted_string_with_embedded_newlines 160 A triple-quoted string with embedded raw newlines round-trips exactly.
function test_roundtrip_nested_fstring_same_quote_pep701 167 PEP 701: a nested string reusing the SAME quote char round-trips.
function test_roundtrip_fstring_debug_specifier 174 The f-string '=' debug specifier (PEP 572/701) round-trips exactly.
function test_roundtrip_mixed_crlf_and_lf 181 A file mixing CRLF and LF newlines round-trips exactly.

tests/refactor/ctok/test_tokenizer.py

source · 20 tests · 235 lines

Tests for orchestrata_ultimate.refactor.ctok.tokenizer.

Covers: refactor.ctok.token_stream, refactor.ctok.tokenizer

Kind Name Line Description
function test_token_kind_has_all_sixteen_members 23 The ratified 13-member enum plus the additive TSTRING_* trio.
function test_kind_by_code_matches_token_kind_order 48 The C extension's integer codes are positional against this tuple.
function test_token_text_returns_exact_byte_span 54 Token.text(source) is source[start:end], nothing else.
function test_token_is_frozen_dataclass 64 Token instances are immutable (frozen, slots).
function test_tokenize_returns_token_stream 71 CTokenizer.tokenize returns a TokenStream over the same source.
function test_tokenize_simple_assignment_kinds_and_text 80 A minimal statement produces NAME, OP, NUMBER, NEWLINE, ENDMARKER.
function test_tokenize_string_kind 101 A plain string literal produces a STRING token with exact text.
function test_tokenize_comment_kind 109 A trailing comment produces a COMMENT token with exact text.
function test_tokenize_comment_only_file 117 A comment-only file still round-trips and yields a COMMENT + NL.
function test_tokenize_indent_dedent 127 A nested block produces INDENT then DEDENT tokens.
function test_tokenize_fstring_kinds 136 PEP 701 f-string: FSTRING_START/MIDDLE/END with a nested expression.
function test_tokenize_tstring_kinds_pep750 156 PEP 750 t-string (new in Python 3.14): TSTRING_START/MIDDLE/END.
function test_tokenize_unicode_identifier 169 A unicode identifier tokenizes as NAME with its exact UTF-8 bytes.
function test_tokenize_soft_keywords_are_name 178 match/case/type are lexed as NAME (they are soft keywords, not OP).
function test_tokenize_empty_source 187 An empty file tokenizes to (at most) an ENDMARKER and round-trips.
function test_tokenize_no_trailing_newline 196 A file with no trailing newline still round-trips exactly.
function test_tokenize_crlf_preserved 203 CRLF line endings are preserved byte-for-byte, never normalized.
function test_tokenize_bom_preserved 212 A leading UTF-8 BOM round-trips even though it is not itself a token.
function test_tokenize_tabs_and_blank_lines 222 Tabs for indentation and interleaved blank lines round-trip exactly.
function test_tokenize_line_continuation 231 An explicit backslash line continuation round-trips exactly.

Support files

Fixtures, package markers and helpers without test functions.