Orchestrata Ultimate › Plans › briefs › kimi-c1-result

Kimi C1 — failure census of 2026-07-22 full-suite run

Source: plans/briefs/kimi-c1-result.md at 8a2cb42ad · last changed 2026-07-27 · first committed 2026-07-22 · 958 lines.

Prose pending.

Plan text

Kimi C1 — failure census of 2026-07-22 full-suite run

Source: plans/briefs/full_suite_run2.json (output of scripts/run_tests.py --json).

Note: the batched runner truncates per-module captured output to the last 6,000 characters. The table below classifies every failure/error that is individually exposed in that output. Exposed failures/errors: 35. The run's headline total is 41 failed + 15 errors = 56; the remaining 21 are hidden by the truncation and are not individually attributable from this artifact.

Totals: port-race = 2, file-dir-constraint = 6, other = 27 Sum of classified rows: 35 (run headline total: 56).

OTHER-class findings

ESCALATE: third failure class present

tests/gateway/test_publisher_wiring.py — tests.gateway.test_publisher_wiring::test_handle_evaluate_with_real_engine_publishes_governance_event

tests.gateway.test_publisher_wiring::test_handle_evaluate_with_real_engine_publishes_governance_event — tests/gateway/test_publisher_wiring.py:365: in test_handle_evaluate_with_real_engine_publishes_governance_event
    assert result["result"] == "denied"  # no rules loaded -> real fail-closed deny
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E   AssertionError: assert 'allowed' == 'denied'
E     
E     - denied
E     + allowed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/gateway/test_publisher_wiring.py:365: in test_handle_evaluate_with_real_engine_publishes_governance_event
    assert result["result"] == "denied"  # no rules loaded -> real fail-closed deny
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E   AssertionError: assert 'allowed' == 'denied'
E     
E     - denied
E     + allowed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/gateway/test_test_runner.py — tests.gateway.test_test_runner::test_run_completes_despite_broadcast_failures

tests.gateway.test_test_runner::test_run_completes_despite_broadcast_failures — tests/gateway/test_test_runner.py:325: in test_run_completes_despite_broadcast_failures
    assert final["status"] == "complete"
E   AssertionError: assert 'failed' == 'complete'
E     
E     - complete
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/gateway/test_test_runner.py:325: in test_run_completes_despite_broadcast_failures
    assert final["status"] == "complete"
E   AssertionError: assert 'failed' == 'complete'
E     
E     - complete
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/gateway/test_test_runner.py — tests.gateway.test_test_runner::test_run_completes_despite_neo4j_store_failure

tests.gateway.test_test_runner::test_run_completes_despite_neo4j_store_failure — tests/gateway/test_test_runner.py:352: in test_run_completes_despite_neo4j_store_failure
    assert final["status"] == "complete"
E   AssertionError: assert 'failed' == 'complete'
E     
E     - complete
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/gateway/test_test_runner.py:352: in test_run_completes_despite_neo4j_store_failure
    assert final["status"] == "complete"
E   AssertionError: assert 'failed' == 'complete'
E     
E     - complete
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/gateway/test_test_runner.py — tests.gateway.test_test_runner::test_run_passes_extra_args_through_to_pytest_command

_args_through_to_pytest_command
    assert final["collected"] == 2
E   assert 0 == 2

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------
tests/gateway/test_test_runner.py:254: in test_run_passes_extra_args_through_to_pytest_command
    assert final["collected"] == 2
E   assert 0 == 2

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------

tests/gateway/test_test_runner.py — tests.gateway.test_test_runner::test_run_records_error_outcome_and_summary_error_count

tests.gateway.test_test_runner::test_run_records_error_outcome_and_summary_error_count — tests/gateway/test_test_runner.py:302: in test_run_records_error_outcome_and_summary_error_count
    assert final["errors"] == 1
E   assert 0 == 1

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------
tests/gateway/test_test_runner.py:302: in test_run_records_error_outcome_and_summary_error_count
    assert final["errors"] == 1
E   assert 0 == 1

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------

tests/gateway/test_test_runner.py — tests.gateway.test_test_runner::test_run_times_out_kills_and_reaps_subprocess

tests.gateway.test_test_runner::test_run_times_out_kills_and_reaps_subprocess — tests/gateway/test_test_runner.py:377: in test_run_times_out_kills_and_reaps_subprocess
    assert final["status"] == "error"
E   AssertionError: assert 'failed' == 'error'
E     
E     - error
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/gateway/test_test_runner.py:377: in test_run_times_out_kills_and_reaps_subprocess
    assert final["status"] == "error"
E   AssertionError: assert 'failed' == 'error'
E     
E     - error
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/gateway/test_test_runner_streaming.py — tests.gateway.test_test_runner_streaming::test_run_failing_reports_failed_status_and_names

tests.gateway.test_test_runner_streaming::test_run_failing_reports_failed_status_and_names — tests/gateway/test_test_runner_streaming.py:228: in test_run_failing_reports_failed_status_and_names
    assert final["passed"] == 1
E   assert 0 == 1

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------
tests/gateway/test_test_runner_streaming.py:228: in test_run_failing_reports_failed_status_and_names
    assert final["passed"] == 1
E   assert 0 == 1

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------

tests/gateway/test_test_runner_streaming.py — tests.gateway.test_test_runner_streaming::test_run_persists_result_to_real_neo4j

tests.gateway.test_test_runner_streaming::test_run_persists_result_to_real_neo4j — tests/gateway/test_test_runner_streaming.py:344: in test_run_persists_result_to_real_neo4j
    assert final["status"] == "complete"
E   AssertionError: assert 'failed' == 'complete'
E     
E     - complete
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/gateway/test_test_runner_streaming.py:344: in test_run_persists_result_to_real_neo4j
    assert final["status"] == "complete"
E   AssertionError: assert 'failed' == 'complete'
E     
E     - complete
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/gateway/test_test_runner_streaming.py — tests.gateway.test_test_runner_streaming::test_run_streams_per_test_progress

tests.gateway.test_test_runner_streaming::test_run_streams_per_test_progress — tests/gateway/test_test_runner_streaming.py:196: in test_run_streams_per_test_progress
    assert any(p.get("current_test") for p in running)
E   assert False
E    +  where False = any(<generator object test_run_streams_per_test_progress.<locals>.<genexpr> at 0x4da603b7b60>)

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------
tests/gateway/test_test_runner_streaming.py:196: in test_run_streams_per_test_progress
    assert any(p.get("current_test") for p in running)
E   assert False
E    +  where False = any(<generator object test_run_streams_per_test_progress.<locals>.<genexpr> at 0x4da603b7b60>)

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------

tests/gateway/test_test_runner_streaming.py — tests.gateway.test_test_runner_streaming::test_run_streams_started_then_complete

ror type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/gateway/test_test_runner_streaming.py:167: in test_run_streams_started_then_complete
    assert progress[-1]["status"] == "complete"
E   AssertionError: assert 'failed' == 'complete'
E     
E     - complete
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/gateway/test_test_runner_streaming.py — tests.gateway.test_test_runner_streaming::test_run_with_real_pubsub_publish_sink

tests.gateway.test_test_runner_streaming::test_run_with_real_pubsub_publish_sink — tests/gateway/test_test_runner_streaming.py:306: in test_run_with_real_pubsub_publish_sink
    assert final["status"] == "complete"
E   AssertionError: assert 'failed' == 'complete'
E     
E     - complete
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/gateway/test_test_runner_streaming.py:306: in test_run_with_real_pubsub_publish_sink
    assert final["status"] == "complete"
E   AssertionError: assert 'failed' == 'complete'
E     
E     - complete
E     + failed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/generated_agents/test_tool_registry_generated.py — tests.generated_agents.test_tool_registry_generated::test_register_native_tools_line_1922_interface

tests.generated_agents.test_tool_registry_generated::test_register_native_tools_line_1922_interface — tests/generated_agents/test_tool_registry_generated.py:916: in test_register_native_tools_line_1922_interface
    assert (
E   AssertionError: Parameter mismatch: expected ['registry'], got ['registry', 'codebase_kb', 'codebase_kb_environ']
E   assert ['registry', ...e_kb_environ'] == ['registry']
E     
E     Left contains 2 more items, first extra item: 'codebase_kb'
E     Use -v to get more diff

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/generated_agents/test_tool_registry_generated.py:916: in test_register_native_tools_line_1922_interface
    assert (
E   AssertionError: Parameter mismatch: expected ['registry'], got ['registry', 'codebase_kb', 'codebase_kb_environ']
E   assert ['registry', ...e_kb_environ'] == ['registry']
E     
E     Left contains 2 more items, first extra item: 'codebase_kb'
E     Use -v to get more diff

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/generated_agents/test_tool_registry_generated.py — tests.generated_agents.test_tool_registry_generated::test_tool_executor___init___line_1033_interface

tests.generated_agents.test_tool_registry_generated::test_tool_executor___init___line_1033_interface — tests/generated_agents/test_tool_registry_generated.py:576: in test_tool_executor___init___line_1033_interface
    assert (
E   AssertionError: Parameter mismatch: expected ['workspace', 'postman', 'from_alias', 'thread_id', 'volk_registry'], got ['workspace', 'postman', 'from_alias', 'thread_id', 'volk_registry', 'codebase_kb', 'codebase_kb_environ']
E   assert ['workspace',...base_kb', ...] == ['workspace',...olk_registry']
E     
E     Left contains 2 more items, first extra item: 'codebase_kb'
E     Use -v to get more diff

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/generated_agents/test_tool_registry_generated.py:576: in test_tool_executor___init___line_1033_interface
    assert (
E   AssertionError: Parameter mismatch: expected ['workspace', 'postman', 'from_alias', 'thread_id', 'volk_registry'], got ['workspace', 'postman', 'from_alias', 'thread_id', 'volk_registry', 'codebase_kb', 'codebase_kb_environ']
E   assert ['workspace',...base_kb', ...] == ['workspace',...olk_registry']
E     
E     Left contains 2 more items, first extra item: 'codebase_kb'
E     Use -v to get more diff

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/generated_agents/test_volks_compliance__pipeline_generated.py — tests.generated_agents.test_volks_compliance__pipeline_generated::test_check_compliance_line_208_interface

tests.generated_agents.test_volks_compliance__pipeline_generated::test_check_compliance_line_208_interface — tests/generated_agents/test_volks_compliance__pipeline_generated.py:269: in test_check_compliance_line_208_interface
    assert (
E   AssertionError: Parameter mismatch: expected ['kg_store', 'task_summary'], got ['kg_store', 'project_dir']
E   assert ['kg_store', 'project_dir'] == ['kg_store', 'task_summary']
E     
E     At index 1 diff: 'project_dir' != 'task_summary'
E     Use -v to get more diff

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/generated_agents/test_volks_compliance__pipeline_generated.py:269: in test_check_compliance_line_208_interface
    assert (
E   AssertionError: Parameter mismatch: expected ['kg_store', 'task_summary'], got ['kg_store', 'project_dir']
E   assert ['kg_store', 'project_dir'] == ['kg_store', 'task_summary']
E     
E     At index 1 diff: 'project_dir' != 'task_summary'
E     Use -v to get more diff

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/generated_agents/test_volks_compliance__pipeline_generated.py — tests.generated_agents.test_volks_compliance__pipeline_generated::test_load_requirements_line_79_interface

tests.generated_agents.test_volks_compliance__pipeline_generated::test_load_requirements_line_79_interface — tests/generated_agents/test_volks_compliance__pipeline_generated.py:97: in test_load_requirements_line_79_interface
    assert (
E   AssertionError: Parameter mismatch: expected ['kg_store', 'task_summary'], got ['kg_store', 'project_dir']
E   assert ['kg_store', 'project_dir'] == ['kg_store', 'task_summary']
E     
E     At index 1 diff: 'project_dir' != 'task_summary'
E     Use -v to get more diff

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/generated_agents/test_volks_compliance__pipeline_generated.py:97: in test_load_requirements_line_79_interface
    assert (
E   AssertionError: Parameter mismatch: expected ['kg_store', 'task_summary'], got ['kg_store', 'project_dir']
E   assert ['kg_store', 'project_dir'] == ['kg_store', 'task_summary']
E     
E     At index 1 diff: 'project_dir' != 'task_summary'
E     Use -v to get more diff

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/generated_agents/test_volks_compliance__pipeline_generated.py — tests.generated_agents.test_volks_compliance__pipeline_generated::test_sync_code_line_175_interface

tests.generated_agents.test_volks_compliance__pipeline_generated::test_sync_code_line_175_interface — tests/generated_agents/test_volks_compliance__pipeline_generated.py:211: in test_sync_code_line_175_interface
    assert (
E   AssertionError: Parameter mismatch: expected ['kg_store', 'task_summary', 'trace_records'], got ['kg_store', 'project_dir', 'trace_records']
E   assert ['kg_store', ...race_records'] == ['kg_store', ...race_records']
E     
E     At index 1 diff: 'project_dir' != 'task_summary'
E     Use -v to get more diff

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/generated_agents/test_volks_compliance__pipeline_generated.py:211: in test_sync_code_line_175_interface
    assert (
E   AssertionError: Parameter mismatch: expected ['kg_store', 'task_summary', 'trace_records'], got ['kg_store', 'project_dir', 'trace_records']
E   assert ['kg_store', ...race_records'] == ['kg_store', ...race_records']
E     
E     At index 1 diff: 'project_dir' != 'task_summary'
E     Use -v to get more diff

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/generated_orchestrator/test_pipeline_kernel_generated.py — tests.generated_orchestrator.test_pipeline_kernel_generated::test_pipeline_kernel_client_start_workflow_line_305_smoke

tests.generated_orchestrator.test_pipeline_kernel_generated::test_pipeline_kernel_client_start_workflow_line_305_smoke — failed on teardown with "Failed: Timeout (>120.0s) from pytest-timeout."
.venv/lib/python3.14t/site-packages/pytest_asyncio/plugin.py:817: in _scoped_runner
    runner.__exit__(None, None, None)
../../.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/asyncio/runners.py:64: in __exit__
    self.close()
../../.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/asyncio/runners.py:74: in close
    loop.run_until_complete(
../../.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/asyncio/base_events.py:706: in run_until_complete
    self.run_forever()
../../.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/asyncio/base_events.py:677: in run_forever
    self._run_once()
../../.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/asyncio/base_events.py:2018: in _run_once
    event_list = self._selector.select(timeout)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/selectors.py:452: in select
    fd_event_list = self._selector.poll(timeout, max_ev)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E   Failed: Timeout (>120.0s) from pytest-timeout.

tests/governance/test_gateway_governance_handler.py — tests.governance.test_gateway_governance_handler::test_handle_evaluate_with_injected_engine

tests.governance.test_gateway_governance_handler::test_handle_evaluate_with_injected_engine — tests/governance/test_gateway_governance_handler.py:41: in test_handle_evaluate_with_injected_engine
    assert result["result"] == "denied"
E   AssertionError: assert 'allowed' == 'denied'
E     
E     - denied
E     + allowed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/governance/test_gateway_governance_handler.py:41: in test_handle_evaluate_with_injected_engine
    assert result["result"] == "denied"
E   AssertionError: assert 'allowed' == 'denied'
E     
E     - denied
E     + allowed

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/knowledge_graph/test_conversation_consumer.py — tests.knowledge_graph.test_conversation_consumer.TestConversationQueueConsumerFlush::test_flush_nonblocking_on_graph_failure

tests.knowledge_graph.test_conversation_consumer.TestConversationQueueConsumerFlush::test_flush_nonblocking_on_graph_failure — tests/knowledge_graph/test_conversation_consumer.py:548: in test_flush_nonblocking_on_graph_failure
    assert redis_client.llen(processing_key) == 1  # left claimed, not lost
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E   AssertionError: assert 0 == 1
E    +  where 0 = llen('_test_1047953e657c:conv:processing')
E    +    where llen = <redis.client.Redis(<redis.connection.ConnectionPool(<redis.connection.Connection(db=0,username=<REDACTED>,password=«redacted»

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/knowledge_graph/test_conversation_consumer.py:548: in test_flush_nonblocking_on_graph_failure
    assert redis_client.llen(processing_key) == 1  # left claimed, not lost
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E   AssertionError: assert 0 == 1
E    +  where 0 = llen('_test_1047953e657c:conv:processing')
E    +    where llen = <redis.client.Redis(<redis.connection.ConnectionPool(<redis.connection.Connection(db=0,username=<REDACTED>,password=«redacted»

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/knowledge_graph/test_neo4j_client_coverage.py — tests.knowledge_graph.test_neo4j_client_coverage::test_ensure_schema_applies_canonical_identity_constraints

tests.knowledge_graph.test_neo4j_client_coverage::test_ensure_schema_applies_canonical_identity_constraints — tests/knowledge_graph/test_neo4j_client_coverage.py:211: in test_ensure_schema_applies_canonical_identity_constraints
    assert not missing, f"Wave-4 canonical constraints not applied: {missing}"
E   AssertionError: Wave-4 canonical constraints not applied: {('Project', 'slug'), ('Session', 'session_id')}
E   assert not {('Project', 'slug'), ('Session', 'session_id')}

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/knowledge_graph/test_neo4j_client_coverage.py:211: in test_ensure_schema_applies_canonical_identity_constraints
    assert not missing, f"Wave-4 canonical constraints not applied: {missing}"
E   AssertionError: Wave-4 canonical constraints not applied: {('Project', 'slug'), ('Session', 'session_id')}
E   assert not {('Project', 'slug'), ('Session', 'session_id')}

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/scripts/test_agent_check.py — tests.scripts.test_agent_check::test_main_listen_then_pull_end_to_end

tderr
E   AssertionError: Listening on Redis channel 'agent:done:_TEST_cf2f7ebe575d_listen'...
E     Received: {'status': 'done'}
E     Pulling code from claude-orchestrata:/home/mohamed/workspace/_TEST_cf2f7ebe575d_listen -> /tmp/pytest-of-mohamed/pytest-441/test_main_listen_then_pull_end0/staging/_TEST_cf2f7ebe575d_listen
E     Code staged at /tmp/pytest-of-mohamed/pytest-441/test_main_listen_then_pull_end0/staging/_TEST_cf2f7ebe575d_listen
E     
E     === Running ruff ===
E     Ruff: 0 issues
E     
E     === Running mypy ===
E     Traceback (most recent call last):
E       File "/home/mohamed/workspace/orchestrata-ultimate-architecture/scripts/agent_check.py", line 333, in <module>
E         main()
E         ~~~~^^
E       File "/home/mohamed/workspace/orchestrata-ultimate-architecture/scripts/agent_check.py", line 304, in main
E         results = run_checks(staged, Path(args.review_script))
E       File "/home/mohamed/workspace/orchestrata-ultimate-architecture/scripts/agent_check.py", line 119, in run_checks
E         mypy = subprocess.run(
E             ["mypy", str(src_path), "--no-error-summary"],
E             capture_output=True,
E             text=True,
E         )
E       File "/home/mohamed/.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/subprocess.py", line 555, in run
E         with Popen(*popenargs, **kwargs) as process:
E              ~~~~~^^^^^^^^^^^^^^^^^^^^^^
E       File "/home/mohamed/.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/subprocess.py", line 1039, in __init__
E         self._execute_child(args, executable, preexec_fn, close_fds,
E         ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E                             pass_fds, cwd, env,
E                             ^^^^^^^^^^^^^^^^^^^
E         ...<5 lines>...
E                             gid, gids, uid, umask,
E                             ^^^^^^^^^^^^^^^^^^^^^^
E                             start_new_session, process_group)
E                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       File "/home/mohamed/.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/subprocess.py", line 1990, in _execute_child
E         raise child_exception_type(errno_num, err_msg, err_filename)
E     FileNotFoundError: [Errno 2] No such file or directory: 'mypy'
E     
E   assert 1 == 0
E    +  where 1 = <Popen: returncode: 1 args: ['/home/mohamed/workspace/orchestrata-ultimate-a...>.returncode

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/scripts/test_agent_check.py:488: in test_main_listen_then_pull_end_to_end
    assert proc.returncode == 0, stdout + stderr
E   AssertionError: Listening on Redis channel 'agent:done:_TEST_cf2f7ebe575d_listen'...
E     Received: {'status': 'done'}
E     Pulling code from claude-orchestrata:/home/mohamed/workspace/_TEST_cf2f7ebe575d_listen -> /tmp/pytest-of-mohamed/pytest-441/test_main_listen_then_pull_end0/staging/_TEST_cf2f7ebe575d_listen
E     Code staged at /tmp/pytest-of-mohamed/pytest-441/test_main_listen_then_pull_end0/staging/_TEST_cf2f7ebe575d_listen
E     
E     === Running ruff ===
E     Ruff: 0 issues
E     
E     === Running mypy ===
E     Traceback (most recent call last):
E       File "/home/mohamed/workspace/orchestrata-ultimate-architecture/scripts/agent_check.py", line 333, in <module>
E         main()
E         ~~~~^^
E       File "/home/mohamed/workspace/orchestrata-ultimate-architecture/scripts/agent_check.py", line 304, in main
E         results = run_checks(staged, Path(args.review_script))
E       File "/home/mohamed/workspace/orchestrata-ultimate-architecture/scripts/agent_check.py", line 119, in run_checks
E         mypy = subprocess.run(
E             ["mypy", str(src_path), "--no-error-summary"],
E             capture_output=True,
E             text=True,
E         )
E       File "/home/mohamed/.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/subprocess.py", line 555, in run
E         with Popen(*popenargs, **kwargs) as process:
E              ~~~~~^^^^^^^^^^^^^^^^^^^^^^
E       File "/home/mohamed/.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/subprocess.py", line 1039, in __init__
E         self._execute_child(args, executable, preexec_fn, close_fds,
E         ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E                             pass_fds, cwd, env,
E                             ^^^^^^^^^^^^^^^^^^^
E         ...<5 lines>...
E                             gid, gids, uid, umask,
E                             ^^^^^^^^^^^^^^^^^^^^^^
E                             start_new_session, process_group)
E                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       File "/home/mohamed/.local/share/uv/python/cpython-3.14.6+freethreaded-linux-x86_64-gnu/lib/python3.14t/subprocess.py", line 1990, in _execute_child
E         raise child_exception_type(errno_num, err_msg, err_filename)
E     FileNotFoundError: [Errno 2] No such file or directory: 'mypy'
E     
E   assert 1 == 0
E    +  where 1 = <Popen: returncode: 1 args: ['/home/mohamed/workspace/orchestrata-ultimate-a...>.returncode

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/scripts/test_code_review_tool.py — tests.scripts.test_code_review_tool::test_mypy_checker_clean_file_no_issues

tests.scripts.test_code_review_tool::test_mypy_checker_clean_file_no_issues — tests/scripts/test_code_review_tool.py:401: in test_mypy_checker_clean_file_no_issues
    assert report.total_issues == 0
E   AssertionError: assert 1 == 0
E    +  where 1 = ReviewReport(files_analyzed=0, total_issues=1, issues=[CodeIssue(file='/tmp/pytest-of-mohamed/pytest-437/test_mypy_che...ltdict(<class 'int'>, {'mypy': 1}), summary_by_severity=defaultdict(<class 'int'>, {<Severity.WARNING: 'WARNING'>: 1})).total_issues

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/scripts/test_code_review_tool.py:401: in test_mypy_checker_clean_file_no_issues
    assert report.total_issues == 0
E   AssertionError: assert 1 == 0
E    +  where 1 = ReviewReport(files_analyzed=0, total_issues=1, issues=[CodeIssue(file='/tmp/pytest-of-mohamed/pytest-437/test_mypy_che...ltdict(<class 'int'>, {'mypy': 1}), summary_by_severity=defaultdict(<class 'int'>, {<Severity.WARNING: 'WARNING'>: 1})).total_issues

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/scripts/test_code_review_tool.py — tests.scripts.test_code_review_tool::test_mypy_checker_detects_type_error

tests.scripts.test_code_review_tool::test_mypy_checker_detects_type_error — tests/scripts/test_code_review_tool.py:388: in test_mypy_checker_detects_type_error
    assert len(mypy_issues) >= 1
E   assert 0 >= 1
E    +  where 0 = len([])

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------
tests/scripts/test_code_review_tool.py:388: in test_mypy_checker_detects_type_error
    assert len(mypy_issues) >= 1
E   assert 0 >= 1
E    +  where 0 = len([])

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------

tests/scripts/test_health_check.py — tests.scripts.test_health_check::test_main_ci_mode_includes_pytest_and_passes

▶ Infra drift (Terraform) ... ✅ PASS
E     
E     ▶ Prolog boundary ... ❌ FAIL
E     
E     ▶ KG schema (advisory) ... 
E       ⚠ ADVISORY (non-blocking) — kg_schema check reports drift:
E         python: can't open file '/tmp/pytest-of-mohamed/pytest-445/test_main_default_mode_include0/scripts/kg_schema.py': [Errno 2] No such file or directory
E       (CI enforces this as a required gate; fix before pushing.)
E     ✅ PASS
E     
E     ============================================================
E     RESULT: 1 ERROR(S)
E     ============================================================
E       PROLOG-BOUNDARY: python: can't open file '/tmp/pytest-of-mohamed/pytest-445/test_main_default_mode_include0/scripts/check_prolog_boundary.py': [Errno 2] No such file or directory
E     
E   assert 1 == 0
E    +  where 1 = CompletedProcess(args=['/home/mohamed/workspace/orchestrata-ultimate-architecture/.venv/bin/python', '/home/mohamed/wo...5/test_main_default_mode_include0/scripts/check_prolog_boundary.py': [Errno 2] No such file or directory\n", stderr='').returncode

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/scripts/test_kg_schema.py — tests.scripts.test_kg_schema.TestCmdCheck::test_check_against_real_shared_graph_conforms

tests.scripts.test_kg_schema.TestCmdCheck::test_check_against_real_shared_graph_conforms — tests/scripts/test_kg_schema.py:235: in test_check_against_real_shared_graph_conforms
    assert exit_code == 0
E   assert 1 == 0

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------
tests/scripts/test_kg_schema.py:235: in test_check_against_real_shared_graph_conforms
    assert exit_code == 0
E   assert 1 == 0

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    Unknown pattern
Likely cause:  Manual investigation needed
See:           docs/DIAGNOSTIC_PLAYBOOK.md
----------------------------------------------------------------------

tests/test_orchestrata_tools.py — tests.test_orchestrata_tools.TestCoverageAnalyzer::test_report_has_gaps

tests.test_orchestrata_tools.TestCoverageAnalyzer::test_report_has_gaps — tests/test_orchestrata_tools.py:220: in test_report_has_gaps
    assert len(report.gaps) < 90
E   AssertionError: assert 95 < 90
E    +  where 95 = len([CoverageGap(method=MethodInfo(name='handle_starttag', qualified_name='agents.tool_registry._DDGResultParser.handle_st...gents.tool_registry', class_name='_HTMLTextExtractor', suggested_test_name='test__htmltextextractor_handle_data'), ...])
E    +    where [CoverageGap(method=MethodInfo(name='handle_starttag', qualified_name='agents.tool_registry._DDGResultParser.handle_st...gents.tool_registry', class_name='_HTMLTextExtractor', suggested_test_name='test__htmltextextractor_handle_data'), ...] = CoverageReport(total_methods=2826, tested_methods=2731, untested_methods=95, method_coverage_pct=96.6, class_coverage_..., 'verification.types': 100.0, 'verification.verification.engine': 100.0, 'verification.verification.feedback': 100.0}).gaps

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/test_orchestrata_tools.py:220: in test_report_has_gaps
    assert len(report.gaps) < 90
E   AssertionError: assert 95 < 90
E    +  where 95 = len([CoverageGap(method=MethodInfo(name='handle_starttag', qualified_name='agents.tool_registry._DDGResultParser.handle_st...gents.tool_registry', class_name='_HTMLTextExtractor', suggested_test_name='test__htmltextextractor_handle_data'), ...])
E    +    where [CoverageGap(method=MethodInfo(name='handle_starttag', qualified_name='agents.tool_registry._DDGResultParser.handle_st...gents.tool_registry', class_name='_HTMLTextExtractor', suggested_test_name='test__htmltextextractor_handle_data'), ...] = CoverageReport(total_methods=2826, tested_methods=2731, untested_methods=95, method_coverage_pct=96.6, class_coverage_..., 'verification.types': 100.0, 'verification.verification.engine': 100.0, 'verification.verification.feedback': 100.0}).gaps

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

tests/test_orchestrata_tools.py — tests.test_orchestrata_tools.TestTestMatcher::test_coverage_matches_legacy_tool

tests.test_orchestrata_tools.TestTestMatcher::test_coverage_matches_legacy_tool — tests/test_orchestrata_tools.py:196: in test_coverage_matches_legacy_tool
    assert len(untested) < 90
E   AssertionError: assert 95 < 90
E    +  where 95 = len([MethodInfo(name='handle_starttag', qualified_name='agents.tool_registry._DDGResultParser.handle_starttag', module_pat...ositional')], return_type='None', docstring=None, class_name='_HTMLTextExtractor', calls=['self._chunks.append']), ...])

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------
tests/test_orchestrata_tools.py:196: in test_coverage_matches_legacy_tool
    assert len(untested) < 90
E   AssertionError: assert 95 < 90
E    +  where 95 = len([MethodInfo(name='handle_starttag', qualified_name='agents.tool_registry._DDGResultParser.handle_starttag', module_pat...ositional')], return_type='None', docstring=None, class_name='_HTMLTextExtractor', calls=['self._chunks.append']), ...])

----------------------------------------------------------------------
DIAGNOSIS
----------------------------------------------------------------------
Error type:    AssertionError
Likely cause:  Test assertion failed — possibly sees data from other tests
Recommended fix: Check if queries filter by test_id, or if assertions are too strict
----------------------------------------------------------------------

Related

• Plans
• Orchestrata Ultimate — project index
• Workflow
• Tasks