orchestrata_ultimate.refactor.ctok.lossless_module

Reference › orchestrata_ultimate › refactor › ctok › lossless_module

LosslessModule ties the layers together: it holds the TokenStream of a file, the standard-library ast.Module parsed from the same bytes, and the NodeIndex that maps the tree to token spans. parse(source) builds all three and the code property reconstructs the bytes, identical to the input (REQ-110-122). Edits return a new immutable module: rename splices new bytes over one token span, replace_span over a byte range, and apply applies an EditBuffer; each splices the bytes and reparses. Only the package facade imports it.

At a glance

Class diagram

Dependency diagram

API

Classes

LosslessModule (dataclass)

Decorators: traces(spec='110', story='US-110-009', reqs=['REQ-110-122', 'REQ-110-123']), dataclass(frozen=True, slots=True) · Line 28

A byte-identical ast + TokenStream + NodeIndex overlay.

Source

Permalink: https://gitlab.com/MSamy/orchestrata-ultimate-architecture/-/blob/a4cbc2bdf/src/orchestrata_ultimate/refactor/ctok/lossless_module.py

"""LosslessModule -- ast + lossless token overlay (P2 of ctok-parser).

P2 of the ctok-parser contract (plans/contracts/ctok-parser.contract.json):
composes P1's :class:`~orchestrata_ultimate.refactor.ctok.tokenizer.
CTokenizer`/:class:`~orchestrata_ultimate.refactor.ctok.token_stream.
TokenStream` with CPython's own ``ast.parse`` and P2's
:class:`~orchestrata_ultimate.refactor.ctok.node_index.NodeIndex` into one
immutable, byte-identical module overlay -- the libcst-free replacement
for the codemod engine's CST layer (REQ-110-122/123). Every edit method
returns a NEW ``LosslessModule`` over the spliced-and-reparsed bytes;
``self`` is never mutated.
"""

from __future__ import annotations

import ast
from dataclasses import dataclass

from orchestrata_ultimate.core.base.traces import traces
from orchestrata_ultimate.refactor.ctok.edit_buffer import EditBuffer
from orchestrata_ultimate.refactor.ctok.node_index import NodeIndex
from orchestrata_ultimate.refactor.ctok.token_stream import TokenStream
from orchestrata_ultimate.refactor.ctok.tokenizer import CTokenizer, Token


@traces(spec="110", story="US-110-009", reqs=["REQ-110-122", "REQ-110-123"])
@dataclass(frozen=True, slots=True)
class LosslessModule:
    """A byte-identical ``ast`` + ``TokenStream`` + ``NodeIndex`` overlay.

    Attributes:
        stream: The lossless TokenStream (P1) over the module's bytes.
        tree: The ``ast.Module`` parsed from the SAME bytes as ``stream``.
        index: NodeIndex mapping ``tree``'s nodes to their token spans.
    """

    stream: TokenStream
    tree: ast.Module
    index: NodeIndex

    @traces(spec="110", story="US-110-009", reqs=["REQ-110-122"])
    @staticmethod
    def parse(source: bytes) -> LosslessModule:
        """Parse ``source`` into a LosslessModule.

        Args:
            source: Raw file bytes.

        Returns:
            A LosslessModule whose ``.code`` is byte-identical to
            ``source`` (REQ-110-122).
        """
        stream = CTokenizer.tokenize(source)
        tree = ast.parse(source)
        index = NodeIndex.build(tree, stream)
        return LosslessModule(stream=stream, tree=tree, index=index)

    @property
    def code(self) -> bytes:
        """The module's bytes -- byte-identical to what it was parsed from."""
        return self.stream.reconstruct()

    def rename(self, span: Token, new_text: bytes) -> LosslessModule:
        """Splice ``new_text`` over ``span`` and reparse.

        Args:
            span: The exact token span to replace, typically from
                ``NodeIndex.token_for_name``.
            new_text: The replacement bytes (e.g. the new identifier).

        Returns:
            A NEW LosslessModule over the spliced bytes; ``self`` is
            unchanged.
        """
        return self.replace_span(span.start, span.end, new_text)

    def replace_span(self, start: int, end: int, new_text: bytes) -> LosslessModule:
        """Splice ``new_text`` over byte range ``[start, end)`` and reparse.

        Args:
            start: Inclusive start byte offset.
            end: Exclusive end byte offset.
            new_text: Replacement bytes.

        Returns:
            A NEW LosslessModule over the spliced bytes; ``self`` is
            unchanged.
        """
        edits = EditBuffer()
        edits.add(start, end, new_text)
        return self.apply(edits)

    @traces(spec="110", story="US-110-009", reqs=["REQ-110-124"])
    def apply(self, edits: EditBuffer) -> LosslessModule:
        """Apply accumulated ``edits`` and reparse the result.

        Args:
            edits: An EditBuffer with one or more accumulated span edits.

        Returns:
            A NEW LosslessModule over the spliced bytes; ``self`` is
            unchanged (immutable edits, REQ-110-124).
        """
        return LosslessModule.parse(edits.apply(self.code))