memory package

Reviewed against commit 8a2cb42ad on 2026-10-23.

Reference: generated package page orchestrata_ultimate.memory (8 modules, 1,754 lines of code); key modules memory_engine, hot_store, cold_store, memory_context. The package's place in the whole system is in the orchestrata overview; the graph it persists to is described on knowledge_graph.

The story

Every agent in the orchestrata pipeline is an LLM behind a stateless API: once a call returns, whatever the volk "knew" is gone. The memory package (src/orchestrata_ultimate/memory/, SPEC-010) exists to give those agents durable, searchable recall. It provides a single unified surface — store, retrieve, search, and summarize — so that volks and the pipeline runner never talk to storage backends directly; they talk to a MemoryEngine.

The core design decision is dual-layer storage. Recent, working memory lives in Redis (hot_store.py, FR-006), where RedisMemoryStore keeps entries in hash structures with a TTL and stores each entry's embedding vector alongside it so similarity search over recent memory stays a single fast round-trip. Long-term memory lives in the operational knowledge graph (cold_store.py, FR-007; the orchkg database on orchgraph, a Memgraph fork — Neo4j was retired on 2026-07-27 and this store was Neo4jMemoryStore until then), where GraphMemoryStore persists each entry as a Memory node and supports scope-filtered and temporal queries. A write flows through MemoryEngine into both layers; retrieval hits the hot layer first for speed and falls back to the durable cold layer for anything that has aged out of Redis. When a session finishes, migrate_to_cold sweeps its hot entries into Neo4j.

Two supporting services make memory more than a key-value store. EmbeddingService (embedding.py, FR-009/FR-010) turns text into fixed-dimension vectors so search can be semantic rather than exact-match — the P1 implementation deliberately uses dependency-free word-frequency (bag-of-words) vectors, each word placed by hash(word) % dimension and the vector L2-normalised (there is no TF-IDF weighting), with sentence-transformers explicitly deferred to P2. ContextCompressor (compressor.py, FR-005/FR-012) squeezes recalled memory into a token budget via extractive summarization, so what gets folded back into an agent's prompt never blows the context window. Budgets are enforced by the provider-aware counters in token_counter.py (FR-011/FR-012), a module shared with the Provider Manager (SPEC-011): WordTokenCounter is the dependency-free approximate default (1 token ≈ 0.75 words), and TiktokenCounter gives exact OpenAI counts when tiktoken is installed, silently falling back to the word counter when it is not.

Within the wider system, the package's neighbors are core (it builds its Redis client through core.redis_factory and carries @traces requirement decorators from core's traceability machinery), the gateway (which owns one shared MemoryEngine instance), and the agents/orchestrator layer, where BaseAgent folds memory recall into its LLM call paths and the pipeline runner binds a memory engine for run-scoped recall. To a new engineer the shape is simple: everything routes through MemoryEngine; the other modules are its two storage layers, its embedder, its compressor, its token accountant, and memory_context.py, the thread-scoped recall bridge the pipelines use.

Key responsibilities

• Unified memory API — one MemoryEngine surface (store / retrieve / search / summarize / update / delete / migrate_to_cold) that hides all storage backends from callers.
• Dual-layer persistence — fast, TTL-bounded working memory in Redis (RedisMemoryStore) and durable long-term memory as Memory nodes in the orchkg graph (GraphMemoryStore), with hot-first, cold-fallback retrieval.
• Semantic search — dependency-free word-frequency embeddings and cosine-similarity ranking (EmbeddingService) over vectors stored next to each entry in both layers.
• Budgeted context compression — extractive summarization (ContextCompressor) that fits recalled memory into a token budget before it is folded into a prompt.
• Provider-aware token accounting — pluggable TokenCounterProtocol implementations (WordTokenCounter, TiktokenCounter) selected via get_token_counter(provider), shared with the Provider Manager.
• Lifecycle & session hygiene — idempotent initialize / start / stop, health/metadata reporting, and end-of-session migration of hot memory to cold storage.

Class diagram

The engine is a composition root: MemoryEngine constructs and owns all five collaborators and is the only class that touches every layer. store writes to the hot store only; entries reach the cold store when migrate_to_cold sweeps them; search ranks candidate embeddings; summarization delegates to the compressor. (Generated from the AST + source, render-verified.)

Token accounting is a small pluggable cluster. Both counters satisfy TokenCounterProtocol; ContextCompressor budgets its summaries through one of them, and TiktokenCounter composes a WordTokenCounter as its fallback when tiktoken is unavailable. (The realization edges are drawn with the inheritance arrow because this renderer rejects the dashed <|.. form; both counters implement the Protocol structurally.)

Module dependency diagram

Internal module imports within orchestrata_ultimate.memory (generated from the AST, render-verified):

Module inventory

memory_engine.py

MemoryEngine implements MemoryEngineProtocol and is the single interface agents use. Its constructor takes Redis and graph connection settings (graph_uri, graph_database and so on, each defaulting from the environment) and wires the rest of the package together: store writes to the Redis hot store (with TTL) only, never directly to the graph cold store; retrieval and semantic search go through the EmbeddingService similarity vectors stored alongside entries; summarization delegates to ContextCompressor, budgeted by a WordTokenCounter. initialize is idempotent (a second call while running is a no-op, to avoid leaking a rebuilt store client), and migrate_to_cold is the one path into the graph: it sweeps a finished session's hot entries into the cold store. Because the engine is the only class that touches all five collaborators, callers elsewhere in the system never need to know which layer a memory came from.

hot_store.py

RedisMemoryStore is the fast working-memory layer (SPEC-010 FR-006). It stores memory entries in Redis hash structures with TTL expiry, so working memory naturally ages out without explicit garbage collection, and it keeps embedding vectors in Redis next to the entries so similarity search over recent memory stays a single fast round-trip. Beyond the basics it offers scope- and tag-filtered retrieval, a global key scan, per-session enumeration, and delete_session for wholesale cleanup. The Redis client itself comes from core.redis_factory, keeping connection configuration consistent with the rest of the codebase.

cold_store.py

GraphMemoryStore (named Neo4jMemoryStore until the 2026-07-27 retirement of Neo4j) is the durable long-term layer (SPEC-010 FR-007). Where the hot store forgets by design, the cold store remembers: entries are persisted as Memory nodes in the operational KG (orchkg on orchgraph, reached over Bolt; "neo4j" survives only as the driver package name) and the store supports scope-filtered search, counting, and temporal queries over the memory history. An available property lets the engine degrade gracefully when the graph is unreachable. This is what lets an agent recall context long after Redis TTLs have expired.

embedding.py

EmbeddingService (SPEC-010 FR-009/FR-010) generates embeddings and computes similarity for semantic memory search, which is what turns retrieval from exact key lookup into "find memories about X". The P1 implementation uses word-frequency (bag-of-words) vectors, hashing each word into a fixed 384-dimension space and normalising the result, with no external dependencies. Python randomises str hashes per process unless PYTHONHASHSEED is fixed, so a vector stored by one process is not comparable with a query vector computed in another, despite the "deterministic" wording in the docstring — a deliberate trade-off that keeps the package installable everywhere — with sentence-transformers integration explicitly deferred to P2. Its search ranks candidate (id, vector) pairs by cosine similarity against a query embedding; both stores hold the vectors this service produces, so the same similarity math works across the hot and cold layers.

compressor.py

ContextCompressor (SPEC-010 FR-005/FR-012) performs token-aware extractive summarization — key-sentence extraction plus truncation — to compress recalled memory into a fixed token budget before it is folded into a prompt. It measures that budget with a counter from token_counter.py, defaulting to WordTokenCounter. LLM-based summarization is deferred to P2, so compression today is deterministic and cheap.

token_counter.py

token_counter.py provides the TokenCounterProtocol implementations used to enforce budgets. WordTokenCounter is the dependency-free approximate default — WORDS_PER_TOKEN = 0.75, so count maps word count to an integer token estimate and truncate clips text to fit. TiktokenCounter wraps tiktoken for exact OpenAI-model counts and holds a WordTokenCounter fallback it uses when tiktoken is not installed. The get_token_counter(provider) factory selects the right counter for a given provider, and the module is shared with the Provider Manager (SPEC-011).

Class map (generated)

Generated from the live source AST by scripts/gen-module-class-diagrams.py — do not hand-edit. Abstract classes have a green border; members listed are those defined on the class itself (inherited members excluded). Solid empty-triangle arrows = in-package inheritance; dashed arrows into dashed boxes = external/stdlib/cross-package bases; filled diamond = composition (constructed in __init__); open diamond = aggregation (annotated/injected reference); diamond edge labels name the owning attribute.