semantic_graph package

Reviewed against commit 8a2cb42ad on 2026-10-23.

Reference: generated package page orchestrata_ultimate.semantic_graph (20 modules including the 3-module adapters subpackage, 5,437 lines of code); key modules interfaces, repository, ingestion, architecture_twin, ha_controller. The overview explains the two graphs and why they must not be confused: orchestrata overview.

The story

Orchestrata's volks need more than raw text search when they reason about a codebase: they need retrieval that understands both meaning (vector similarity over embedded chunks) and structure (entities and relationships in a graph). orchestrata_ultimate.semantic_graph exists to provide exactly that — a repository abstraction for RAG / GraphRAG. Rather than binding the rest of the system to a particular database, the package defines small typing.Protocol contracts in interfaces.py — VectorStore for embedding search, GraphStore for entity/relationship traversal, and SemanticGraph, which inherits both — together with the shared dataclasses in models.py (Entity, Relationship, Chunk, and the GraphRAGContext bundle returned by retrieve). Everything upstream depends only on these contracts, so the concrete backend can change without touching a caller.

Two swappable backends satisfy the contract. adapters/memgraph.py's MemgraphSemanticGraph is the production adapter, speaking Cypher over Bolt to a Memgraph instance, driving a native vector index for search_similar, and fusing that with a keyword scan (search_keyword) and module/entity-prominence boosts for hybrid retrieval; adapters/memory.py's InMemorySemanticGraph is a pure-Python adapter whose vector search is a local cosine similarity, making it the default for unit tests and local development. Callers never choose an adapter directly: repository.py exposes create_semantic_graph(), a factory that reads a SemanticGraphConfig (config.py, resolved from environment variables with no hard-coded IPs) and returns whichever backend the configuration names, raising the package's own exception hierarchy (exceptions.py) on an unknown backend or connection trouble.

Data flows into the graph along two pipelines. The first is code ingestion: ingestion.py's CodeChunker splits source files into overlapping line-window chunks, and SemanticGraphIngester embeds them through a pluggable Embedder backend (embedders.py offers OpenRouterEmbedder, cloud embeddings via OpenRouter/Codestral, and SentenceTransformerEmbedder, local GPU embeddings via sentence-transformers) before upserting chunks, entities, and their relationships into a SemanticGraph (which embedder is used is now decided once, from the environment, by embedder_factory.py; ArchitectureTwin in architecture_twin.py reuses it to index ADRs and design documents). On the way back out, SemanticGraphIngester.search runs hybrid vector+keyword retrieval and can hand its candidates to rerankers.py's OpenRouterReranker for a final relevance re-scoring. The second pipeline is spec/trace enrichment: spec_trace_parser.py parses Orchestrata's SPEC markdown and the @traces annotations scattered through the code (leaning on core.traceability) into SpecInfo / StoryInfo / RequirementInfo structures, and spec_trace_enricher.py's SpecTraceEnricher writes the resulting spec, story, requirement, and code-element entities — plus the trace edges between them — into the graph, returning an EnrichmentSummary of what it wrote.

The package collaborates with its neighbors at arm's length: scripts/ingest_orchestrata_ultimate.py and scripts/enrich_semantic_graph_specs.py drive the two write pipelines from the repo, and core.traceability supplies the @traces machinery the enrichment path reads. Because production Memgraph runs as Community edition without built-in HA, the package also ships its own operational layer: ha_controller.py's MemgraphHAController manages main/replica replication, per-node health state, and failover across a cluster (modeled by NodeConfig, NodeState, and the ClusterStatus snapshot it serves over a small FastAPI surface), ha_controller_cli.py wraps it in an async argparse CLI whose node list can come from the command line or the environment, and main_discovery.py's resolve_main() lets clients discover the current write leader — via the HA controller or by probing the Raft coordinators directly — so writes always land on the live MAIN.

Key responsibilities

• Define backend-agnostic RAG/GraphRAG contracts (VectorStore, GraphStore, SemanticGraph) and the shared data models (Entity, Relationship, Chunk, GraphRAGContext).
• Provide two interchangeable adapters — Memgraph (production, Bolt + native vector index + hybrid retrieval) and in-memory (cosine similarity, for tests/local dev) — selected by create_semantic_graph() from a SemanticGraphConfig.
• Embed and rerank text through pluggable backends (OpenRouterEmbedder, SentenceTransformerEmbedder, OpenRouterReranker) behind a uniform async interface.
• Ingest source code: chunk files (CodeChunker), embed, upsert, and serve hybrid vector+keyword search (SemanticGraphIngester).
• Enrich the graph with specs, stories, requirements, and @traces edges parsed from the repo (spec_trace_parser + SpecTraceEnricher).
• Operate Memgraph HA in the absence of Enterprise clustering: replication management, health polling, and failover (MemgraphHAController) plus write-leader discovery (resolve_main).

Class diagram

Contracts and data models

The Protocol contracts live in interfaces.py; SemanticGraph composes the VectorStore and GraphStore capabilities into the single interface every adapter implements. retrieve returns a GraphRAGContext — the bundle of entities, relationships, and chunks assembled for an LLM prompt.

Backends and the factory

create_semantic_graph() reads a SemanticGraphConfig and returns one of the two adapters. Both realize the SemanticGraph Protocol; the Memgraph adapter adds hybrid-retrieval helpers (search_keyword, link_chunks_to_modules) on top of the base contract.

Embedding, reranking, and ingestion pipeline

SemanticGraphIngester orchestrates the write path: it chunks with CodeChunker, embeds through an Embedder (one of two concrete backends), writes to a SemanticGraph, and optionally reranks retrieval candidates with OpenRouterReranker.

Spec / trace enrichment

The parser (spec_trace_parser.py) turns SPEC markdown into a SpecInfo tree of stories and requirements; SpecTraceEnricher reads those, joins them with collected @traces, writes the resulting entities and edges into a SemanticGraph, and reports an EnrichmentSummary.

Memgraph HA control plane

Because Community-edition Memgraph has no built-in HA, MemgraphHAController runs the replication/failover loop over a fleet of data instances. It holds one NodeState per instance (each wrapping a NodeConfig) and exposes a ClusterStatus snapshot through its FastAPI surface.

Module dependency diagram

Internal module imports within orchestrata_ultimate.semantic_graph (render-verified).

Module inventory

Related

• Semantic graph & RAG — this package owns the shared semantic / RAG graph, the memgraph database on Memgraph Community. It is distinct from the operational KG maintained by knowledge_graph, which is the orchkg database on orchgraph, a Memgraph fork. Neo4j, which both graphs used to be compared against, was retired on 2026-07-27; "neo4j" survives only as the name of the Bolt driver package. The two graphs meet in architecture_twin (reads orchkg, writes memgraph) and in knowledge_graph.semantic_bridge.
• Driver scripts: scripts/ingest_orchestrata_ultimate.py (code ingestion), scripts/enrich_semantic_graph_specs.py (spec/trace enrichment), scripts/evaluate_semantic_graph_qa.py (retrieval eval).

Class map (generated)

Generated from the live source AST by scripts/gen-module-class-diagrams.py — do not hand-edit. Abstract classes have a green border; members listed are those defined on the class itself (inherited members excluded). Solid empty-triangle arrows = in-package inheritance; dashed arrows into dashed boxes = external/stdlib/cross-package bases; filled diamond = composition (constructed in __init__); open diamond = aggregation (annotated/injected reference); diamond edge labels name the owning attribute.