Dedicated Test Memgraph

The problem

MEMGRAPH_URI names the production orchgraph HA cluster — shared with the knowledge-wiki project. Every semantic-graph test creates a uniqueness constraint, two lookup indexes and a vector index for its chunk label against whatever URI resolves, and clear() / clear_prefixed() delete nodes only.

Where a fixture built its label from the test_id uuid, that schema grew without bound and Memgraph replays every entry at boot recovery:

The 2026-07-15 cleanup was ad-hoc, which is exactly why it regrew unnoticed — there was nothing to re-run and nothing to measure.

The fix, in three parts

1. Target. TEST_MEMGRAPH_URI names a dedicated standalone node. tests/conftest.py rewrites MEMGRAPH_URI to it at import time (and blanks MEMGRAPH_URIS / MEMGRAPH_HA_CONTROLLER_URL), which redirects code that resolves the URI for itself — SemanticGraphConfig.from_env, the ingest scripts — that being the larger half of the write paths. tests/infrastructure resolves the test URI first, ahead of the ambient value, so a module imported outside conftest cannot fall through to prod.

2. Source. Two architecture-twin fixtures never called drop_chunk_label_schema(); three siblings already did. Both now drop.

3. Recurrence. test-review rule T17 fails any fixture that builds a uuid-scoped chunk_label without dropping its schema, and scripts/purge_memgraph_test_schema.py is the repeatable cleanup (dry-run by default, --apply to drop, idempotent). Both carry tests.

T17 deliberately does not fire on the bounded per-worker labels (TestChunkgw0) — those keep their schema so the warm-run connect() short-circuit stays effective — nor on pure config factories whose consumers own teardown.

The node

memgraph-test @ 10.53.1.167:7687, no auth, standalone (not a Raft member).

It runs the same build as production — binary sha256 verified equal. The stock Memgraph 3.11.0 deb is not good enough: multi-tenancy (CREATE DATABASE) is license-gated in it, and the vector_search query module is absent. The suite depends on both. Prod's own licence key is invalid; the orchgraph fork is compiled MG_ENTERPRISE=ON, which is what actually unlocks multi-tenancy.

Arch Linux image on purpose, not for symmetry: the fork binary links Arch's glibc 2.43 / libpython3.14 and will not even load on Ubuntu.

Modeled in Terraform (infra/terraform/memgraph_test.tf) rather than left as hand-created drift.

Provision / re-provision

terraform -chdir=infra/terraform apply     # creates the Arch container
incus/orchgraph-test-node-deploy.sh        # syncs prod's /opt/memgraph, unit, databases

The deploy script is idempotent. It also creates the orchkg + orchkg_w0..w7 database topology (derived from the NEO4J_URI_GW<N> keys, the same rule _memgraph_worker_count() uses) — without it the suite fails with Tried to retrieve an unknown database "orchkg".

Main discovery is the one exception

resolve_main() needs a real Raft cluster, which a standalone node is not. Three tests read the production coordinators via TEST_MEMGRAPH_COORDINATOR_URIS — SHOW INSTANCES, then SHOW REPLICATION ROLE. Read-only. Reads never polluted anything; writes did.

Defects the switch exposed

load_dotenv(..., override=True) in the four Memgraph-facing scripts (ingest_orchestrata_ultimate, semantic_search, enrich_semantic_graph_specs, evaluate_semantic_graph_qa) silently replaced an explicitly-exported variable with .env's value. Redirecting these scripts by environment — the entire mechanism keeping the suite off production — therefore did nothing, with no diagnostic. A bootstrap aimed at the test node wrote 592 chunks into the production cluster instead; only a before/after node count caught it. Now override=False, pinned by a test.

graph_browser q filter ran toString(n[prop]) over every property. toString rejects lists, so filtering any label whose nodes carry an embedding vector returned a 500. Now restricted to string properties via valueType() — the IS :: STRING predicate does not parse on this build.

graph_browser hardcoded bolt://localhost:7687 when no env var was set. Nothing here runs Memgraph on localhost, so it could only turn a missing-config bug into a confusing failure against a host that was never the target. Raises now.

Known open item

ingest_orchestrata_ultimate.py stalls at exactly 4096 chunks on a fresh node (2^12; index capacity is 262144, so not a capacity limit). Both client and server sit idle — a deadlock, not slow work, consistent with a vector-index resize boundary in the fork. The test node currently holds the 4096 CodestralChunk that did land, which is enough for the live-graph tests to pass. Not yet root-caused.

Verification

After a full semantic-graph run against the test node, production node/index/constraint counts were unchanged (+0/+0/+0) and the test node held zero uuid-scoped labels. Prod went 484 → 207 indexes and 898 → 75 constraints after the purge, with all five production vector indexes intact at full size.