Orchestrata Ultimate / Specs / 002-postman
002-postman: Postman Message Broker (U:PS)
The Postman (U:PS) is the message routing engine for Orchestrata. It handles message delivery, topic-based subscriptions, queue management, priority processing, message filtering, persistence, and delivery guarantees.
At a glance
| Item | Value |
|---|---|
| Spec directory | specs/002-postman/ |
| Status line in the spec | Complete |
| Requirement IDs defined | 457 |
With a @traces citation in src/ |
227 (49.7%) |
With a @traces citation in tests/ |
13 (2.8%) |
| Tracked files | 8 |
| RTM page | 002-postman |
Reading this spec
The Postman (U:PS) is the message routing engine of Orchestrata. Agents do not talk to each other directly; the Postman handles message delivery, topic-based subscriptions, queue management, priority processing, message filtering, persistence and delivery guarantees, so that communication is reliable and decoupled.
The scope is ten user stories: agent message delivery, topic subscription, queue management, priority handling, filtering and routing, persistence and replay, delivery guarantees and acknowledgments, monitoring and metrics, configuration management, and performance and scalability. Delivery, topic subscription, delivery guarantees and performance are marked critical. Requirement IDs take the forms REQ-002-US-nnn and REQ-002-FRnn; the functional group REQ-002-FR11 is the largest.
The key decisions in the overview are Redis-backed queues and topics, several delivery protocols (CXP, gRPC, WebSocket and HTTP), priority-based processing, and acknowledgment-based delivery. The spec names three delivery guarantees: at-most-once, at-least-once and exactly-once. Persistence supports recovery and replay. The stated performance target is more than 5000 messages per second.
In the code, the package is orchestrata_ultimate/postman. Most citations land in orchestrata_ultimate/postman/interfaces and orchestrata_ultimate/postman/types, with orchestrata_ultimate/postman/priority, orchestrata_ultimate/postman/router and orchestrata_ultimate/postman/queue behind them. The per-requirement view is on the RTM page.
The spec's status line reads Complete. It tracks 8 files and defines 457 requirement IDs. The full text follows below.
Measured coverage is 227 of 457 requirements cited by at least one @traces in src/ (49.7%), spread over 36 modules; the module citing the most of them is orchestrata_ultimate/postman/interfaces (81 requirements). In tests/, 13 requirements are cited by a test-side @traces.
Coverage gaps: 230 requirements have no citing symbol. The largest untraced groups are REQ-002-FR11 (37), REQ-002-US-010 (24), REQ-002-US-008 (21). Untraced means no @traces names the ID; the behaviour may still exist in code that was never annotated. Test-side citations are thinner than code-side ones (13 against 227).
Requirements
Checkbox is the state recorded in the requirements checklist; Traced by is the count of src/ symbols whose @traces names the ID.
| ID | Done | Requirement | Traced by |
|---|---|---|---|
REQ-002-US-001-001 |
yes | Agent can send messages through Postman service | 4 |
REQ-002-US-001-002 |
yes | Messages are routed to correct recipient agents | 5 |
REQ-002-US-001-003 |
yes | Message delivery is acknowledged with receipts | 2 |
REQ-002-US-001-004 |
yes | Failed deliveries trigger retry mechanisms | 4 |
REQ-002-US-001-005 |
yes | Delivery status is tracked and logged | 10 |
REQ-002-US-001-006 |
yes | Support for INFORM message type | 1 |
REQ-002-US-001-007 |
yes | Support for REQUEST message type | 1 |
REQ-002-US-001-008 |
yes | Support for QUERY message type | 1 |
REQ-002-US-001-009 |
yes | Support for RESPONSE message type | 1 |
REQ-002-US-001-010 |
yes | Support for ERROR message type | 1 |
REQ-002-US-001-011 |
yes | Support for NOTIFY message type | 1 |
REQ-002-US-001-012 |
yes | Support for SUBSCRIBE message type | 1 |
REQ-002-US-001-013 |
yes | Support for UNSUBSCRIBE message type | 1 |
REQ-002-US-001-014 |
yes | Direct agent-to-agent delivery | none |
REQ-002-US-001-015 |
yes | Broadcast delivery to multiple agents | none |
REQ-002-US-001-016 |
yes | Topic-based delivery | 2 |
REQ-002-US-001-017 |
yes | Queue-based delivery | none |
REQ-002-US-001-018 |
yes | Priority-based delivery | none |
REQ-002-US-001-019 |
yes | Handle recipient not found errors | none |
REQ-002-US-001-020 |
yes | Handle network timeout errors | none |
REQ-002-US-001-021 |
yes | Handle malformed message errors | none |
REQ-002-US-001-022 |
yes | Handle authentication errors | none |
REQ-002-US-001-023 |
yes | Handle authorization errors | none |
REQ-002-US-001-024 |
yes | Retry failed deliveries (up to 3 attempts) | 5 |
REQ-002-US-001-025 |
yes | Log all delivery failures | none |
REQ-002-US-001-026 |
yes | Notify sender of permanent failures | 1 |
REQ-002-US-001-027 |
yes | Deliver messages within 100ms (p95) | none |
REQ-002-US-001-028 |
yes | Support 5000+ messages/second throughput | none |
REQ-002-US-001-029 |
yes | Handle concurrent deliveries efficiently | none |
REQ-002-US-001-030 |
yes | Maintain message ordering per sender-recipient pair | none |
REQ-002-US-002-001 |
yes | Agent can subscribe to topics | 4 |
REQ-002-US-002-002 |
yes | Agent can unsubscribe from topics | 3 |
REQ-002-US-002-003 |
yes | Agent can list subscribed topics | 1 |
REQ-002-US-002-004 |
yes | Support wildcard topic patterns (e.g., "agent.*") | none |
REQ-002-US-002-005 |
yes | Support hierarchical topic structures | none |
REQ-002-US-002-006 |
yes | Validate topic names on subscription | 1 |
REQ-002-US-002-007 |
yes | Deliver messages to all topic subscribers | 2 |
REQ-002-US-002-008 |
yes | Filter messages by topic match | none |
REQ-002-US-002-009 |
yes | Support multiple simultaneous subscriptions | 1 |
REQ-002-US-002-010 |
yes | Handle subscription conflicts gracefully | none |
REQ-002-US-002-011 |
yes | Maintain delivery order per topic | none |
REQ-002-US-002-012 |
yes | Persist subscriptions across restarts | none |
REQ-002-US-002-013 |
yes | Auto-unsubscribe on agent termination | none |
REQ-002-US-002-014 |
yes | Track subscription creation timestamp | 1 |
REQ-002-US-002-015 |
yes | Track last message received timestamp | 1 |
REQ-002-US-002-016 |
yes | Support subscription TTL (time-to-live) | 3 |
REQ-002-US-002-017 |
yes | Subscribe/unsubscribe within 50ms | none |
REQ-002-US-002-018 |
yes | Deliver to 100+ subscribers per topic | none |
REQ-002-US-002-019 |
yes | Handle 1000+ active subscriptions | none |
REQ-002-US-002-020 |
yes | Efficient topic matching algorithm | none |
REQ-002-US-003-001 |
yes | Create named message queues | 3 |
REQ-002-US-003-002 |
yes | Delete message queues | 3 |
REQ-002-US-003-003 |
yes | List available queues | 3 |
REQ-002-US-003-004 |
yes | Query queue metadata (size, age, etc.) | 6 |
REQ-002-US-003-005 |
yes | Purge queue contents | 3 |
REQ-002-US-003-006 |
yes | FIFO (First-In-First-Out) ordering | 1 |
REQ-002-US-003-007 |
yes | Priority queue support | none |
REQ-002-US-003-008 |
yes | Message deduplication | 2 |
REQ-002-US-003-009 |
yes | Dead-letter queue for failed messages | 3 |
REQ-002-US-003-010 |
yes | Message TTL (time-to-live) | 2 |
REQ-002-US-003-011 |
yes | Single consumer mode | none |
REQ-002-US-003-012 |
yes | Multiple consumer mode (load balancing) | none |
REQ-002-US-003-013 |
yes | Peek messages without consuming | 1 |
REQ-002-US-003-014 |
yes | Batch message retrieval | none |
REQ-002-US-003-015 |
yes | Acknowledge message consumption | 2 |
REQ-002-US-003-016 |
yes | Maximum queue size (10000 messages) | 2 |
REQ-002-US-003-017 |
yes | Maximum message age (24 hours) | none |
REQ-002-US-003-018 |
yes | Queue overflow handling | none |
REQ-002-US-003-019 |
yes | Memory limit per queue | none |
REQ-002-US-003-020 |
yes | Persist queue state to disk | none |
REQ-002-US-003-021 |
yes | Restore queues on restart | none |
REQ-002-US-003-022 |
yes | Handle corruption gracefully | none |
REQ-002-US-003-023 |
yes | Periodic checkpoint creation | none |
REQ-002-US-004-001 |
yes | CRITICAL priority level (0) | 1 |
REQ-002-US-004-002 |
yes | HIGH priority level (1) | 1 |
REQ-002-US-004-003 |
yes | NORMAL priority level (2) | 1 |
REQ-002-US-004-004 |
yes | LOW priority level (3) | 1 |
REQ-002-US-004-005 |
yes | BACKGROUND priority level (4) | 1 |
REQ-002-US-004-006 |
yes | Process messages in priority order | 2 |
REQ-002-US-004-007 |
yes | Preempt lower priority messages | none |
REQ-002-US-004-008 |
yes | Prevent starvation of low-priority messages | 2 |
REQ-002-US-004-009 |
yes | Age-based priority boost | 2 |
REQ-002-US-004-010 |
yes | Priority inversion detection | 1 |
REQ-002-US-004-011 |
yes | Set default priority per agent | none |
REQ-002-US-004-012 |
yes | Override priority per message | 3 |
REQ-002-US-004-013 |
yes | Configure priority thresholds | none |
REQ-002-US-004-014 |
yes | Configure aging parameters | 1 |
REQ-002-US-004-015 |
yes | Track processing time per priority | none |
REQ-002-US-004-016 |
yes | Monitor priority distribution | 1 |
REQ-002-US-004-017 |
yes | Alert on priority queue buildup | none |
REQ-002-US-004-018 |
yes | Report priority inversion incidents | none |
REQ-002-US-005-001 |
yes | Content-based filtering | 6 |
REQ-002-US-005-002 |
yes | Header-based filtering | 3 |
REQ-002-US-005-003 |
yes | Sender-based filtering | 3 |
REQ-002-US-005-004 |
yes | Topic-based filtering | 3 |
REQ-002-US-005-005 |
yes | Pattern-based filtering (regex) | none |
REQ-002-US-005-006 |
yes | Composite filters (AND/OR logic) | 8 |
REQ-002-US-005-007 |
yes | Static routing by recipient | 1 |
REQ-002-US-005-008 |
yes | Dynamic routing by content | none |
REQ-002-US-005-009 |
yes | Conditional routing (if-then rules) | none |
REQ-002-US-005-010 |
yes | Load-balanced routing | none |
REQ-002-US-005-011 |
yes | Failover routing | none |
REQ-002-US-005-012 |
yes | Create routing rules | 4 |
REQ-002-US-005-013 |
yes | Update routing rules | 3 |
REQ-002-US-005-014 |
yes | Delete routing rules | 4 |
REQ-002-US-005-015 |
yes | List active rules | 3 |
REQ-002-US-005-016 |
yes | Enable/disable rules | 1 |
REQ-002-US-005-017 |
yes | Validate rule syntax | 2 |
REQ-002-US-005-018 |
yes | Evaluate rules in priority order | 3 |
REQ-002-US-005-019 |
yes | Stop on first match (if configured) | 2 |
REQ-002-US-005-020 |
yes | Evaluate all matching rules | 1 |
REQ-002-US-005-021 |
yes | Handle rule conflicts | none |
REQ-002-US-005-022 |
yes | Log rule evaluations | none |
REQ-002-US-005-023 |
yes | Rule evaluation < 10ms per message | none |
REQ-002-US-005-024 |
yes | Support 1000+ active rules | none |
REQ-002-US-005-025 |
yes | Efficient pattern matching | 2 |
REQ-002-US-005-026 |
yes | Cache compiled rule expressions | none |
REQ-002-US-006-001 |
yes | Persist all messages to disk | 4 |
REQ-002-US-006-002 |
yes | Store message metadata (timestamp, sender, etc.) | 2 |
REQ-002-US-006-003 |
yes | Store message payload | 2 |
REQ-002-US-006-004 |
yes | Store delivery receipts | none |
REQ-002-US-006-005 |
yes | Compress stored messages | none |
REQ-002-US-006-006 |
yes | Configure retention period (default 7 days) | none |
REQ-002-US-006-007 |
yes | Configure storage limits (default 10GB) | none |
REQ-002-US-006-008 |
yes | Automatic cleanup of old messages | 2 |
REQ-002-US-006-009 |
yes | Manual purge operations | 2 |
REQ-002-US-006-010 |
yes | Export messages to archive | none |
REQ-002-US-006-011 |
yes | Replay messages by time range | 2 |
REQ-002-US-006-012 |
yes | Replay messages by sender | 2 |
REQ-002-US-006-013 |
yes | Replay messages by topic | 1 |
REQ-002-US-006-014 |
yes | Replay messages by message ID | 4 |
REQ-002-US-006-015 |
yes | Replay messages by filter criteria | 3 |
REQ-002-US-006-016 |
yes | Replay at original speed | none |
REQ-002-US-006-017 |
yes | Replay at accelerated speed | 1 |
REQ-002-US-006-018 |
yes | Replay with delay between messages | 3 |
REQ-002-US-006-019 |
yes | Replay to different recipients | 1 |
REQ-002-US-006-020 |
yes | Dry-run replay (no delivery) | 1 |
REQ-002-US-006-021 |
yes | Recover from storage corruption | none |
REQ-002-US-006-022 |
yes | Rebuild indexes from logs | none |
REQ-002-US-006-023 |
yes | Verify message integrity | none |
REQ-002-US-006-024 |
yes | Handle missing messages gracefully | none |
REQ-002-US-007-001 |
yes | At-most-once delivery (fire and forget) | 3 |
REQ-002-US-007-002 |
yes | At-least-once delivery (with retries) | 2 |
REQ-002-US-007-003 |
yes | Exactly-once delivery (with deduplication) | 3 |
REQ-002-US-007-004 |
yes | Configure delivery mode per message | 2 |
REQ-002-US-007-005 |
yes | Configure delivery mode per queue | none |
REQ-002-US-007-006 |
yes | Synchronous ACK (immediate response) | none |
REQ-002-US-007-007 |
yes | Asynchronous ACK (deferred response) | none |
REQ-002-US-007-008 |
yes | Negative ACK (NACK) for failures | 2 |
REQ-002-US-007-009 |
yes | Batch acknowledgments | none |
REQ-002-US-007-010 |
yes | Selective acknowledgments | none |
REQ-002-US-007-011 |
yes | Configure ACK timeout per message | none |
REQ-002-US-007-012 |
yes | Configure ACK timeout per agent | none |
REQ-002-US-007-013 |
yes | Retry on timeout (up to max retries) | none |
REQ-002-US-007-014 |
yes | Exponential backoff for retries | 1 |
REQ-002-US-007-015 |
yes | Alert on persistent timeouts | none |
REQ-002-US-007-016 |
yes | Generate unique receipt ID | 1 |
REQ-002-US-007-017 |
yes | Track receipt status (pending/acked/nacked) | none |
REQ-002-US-007-018 |
yes | Query receipt status by message ID | 1 |
REQ-002-US-007-019 |
yes | Store receipts for audit trail | 2 |
REQ-002-US-007-020 |
yes | Expire old receipts | 1 |
REQ-002-US-007-021 |
yes | Dead-letter queue for undeliverable messages | none |
REQ-002-US-007-022 |
yes | Alert on delivery failures | none |
REQ-002-US-007-023 |
yes | Log detailed failure reasons | none |
REQ-002-US-007-024 |
yes | Retry with different routes | none |
REQ-002-US-007-025 |
yes | Manual intervention support | none |
REQ-002-US-008-001 |
yes | Messages sent per second | 5 |
REQ-002-US-008-002 |
yes | Messages received per second | 5 |
REQ-002-US-008-003 |
yes | Messages queued (current) | 1 |
REQ-002-US-008-004 |
yes | Messages in-flight (current) | none |
REQ-002-US-008-005 |
yes | Messages failed (total) | 5 |
REQ-002-US-008-006 |
yes | Average message latency (p50, p95, p99) | 4 |
REQ-002-US-008-007 |
yes | Message throughput (bytes/sec) | none |
REQ-002-US-008-008 |
yes | Active agents count | 1 |
REQ-002-US-008-009 |
yes | Messages per agent | none |
REQ-002-US-008-010 |
yes | Agent message latency | none |
REQ-002-US-008-011 |
yes | Agent error rate | none |
REQ-002-US-008-012 |
yes | Agent connection status | none |
REQ-002-US-008-013 |
yes | Queue depth per queue | 6 |
REQ-002-US-008-014 |
yes | Queue age (oldest message) | 2 |
REQ-002-US-008-015 |
yes | Messages enqueued per queue | none |
REQ-002-US-008-016 |
yes | Messages dequeued per queue | none |
REQ-002-US-008-017 |
yes | Queue overflow events | none |
REQ-002-US-008-018 |
yes | Subscribers per topic | none |
REQ-002-US-008-019 |
yes | Messages per topic | none |
REQ-002-US-008-020 |
yes | Topic message rate | none |
REQ-002-US-008-021 |
yes | Topic latency | none |
REQ-002-US-008-022 |
yes | CPU usage | none |
REQ-002-US-008-023 |
yes | Memory usage | none |
REQ-002-US-008-024 |
yes | Disk I/O | none |
REQ-002-US-008-025 |
yes | Network I/O | none |
REQ-002-US-008-026 |
yes | Thread pool utilization | none |
REQ-002-US-008-027 |
yes | Prometheus format export | 3 |
REQ-002-US-008-028 |
yes | JSON format export | 4 |
REQ-002-US-008-029 |
yes | CSV format export | none |
REQ-002-US-008-030 |
yes | Real-time streaming metrics | none |
REQ-002-US-008-031 |
yes | Historical metrics query | none |
REQ-002-US-009-001 |
yes | Load from YAML file | 2 |
REQ-002-US-009-002 |
yes | Load from environment variables | none |
REQ-002-US-009-003 |
yes | Load from command-line arguments | none |
REQ-002-US-009-004 |
yes | Configuration priority order | none |
REQ-002-US-009-005 |
yes | Validate configuration on load | 1 |
REQ-002-US-009-006 |
yes | Postman service port | 1 |
REQ-002-US-009-007 |
yes | Maximum message size | 1 |
REQ-002-US-009-008 |
yes | Maximum queue size | 2 |
REQ-002-US-009-009 |
yes | Default message TTL | 1 |
REQ-002-US-009-010 |
yes | Retry policy configuration | 1 |
REQ-002-US-009-011 |
yes | Thread pool size | 1 |
REQ-002-US-009-012 |
yes | Storage path configuration | 2 |
REQ-002-US-009-013 |
yes | Default topic patterns | none |
REQ-002-US-009-014 |
yes | Topic retention policy | 1 |
REQ-002-US-009-015 |
yes | Topic access control | none |
REQ-002-US-009-016 |
yes | Topic rate limits | none |
REQ-002-US-009-017 |
yes | Default queue behavior (FIFO/priority) | none |
REQ-002-US-009-018 |
yes | Queue persistence settings | 1 |
REQ-002-US-009-019 |
yes | Queue size limits | 2 |
REQ-002-US-009-020 |
yes | Dead-letter queue settings | 1 |
REQ-002-US-009-021 |
yes | Enable/disable authentication | none |
REQ-002-US-009-022 |
yes | Enable/disable encryption | none |
REQ-002-US-009-023 |
yes | Certificate paths | none |
REQ-002-US-009-024 |
yes | Access control policies | none |
REQ-002-US-009-025 |
yes | Reload configuration without restart | 3 |
REQ-002-US-009-026 |
yes | Validate configuration changes | none |
REQ-002-US-009-027 |
yes | Rollback failed changes | none |
REQ-002-US-009-028 |
yes | Notify on configuration changes | 5 |
REQ-002-US-010-001 |
yes | Handle 5000+ messages/second | none |
REQ-002-US-010-002 |
yes | Handle 10000+ concurrent connections | none |
REQ-002-US-010-003 |
yes | Handle 1000+ active queues | none |
REQ-002-US-010-004 |
yes | Handle 100+ topics with subscribers | none |
REQ-002-US-010-005 |
yes | Message delivery < 100ms (p95) | none |
REQ-002-US-010-006 |
yes | Queue operations < 50ms (p95) | none |
REQ-002-US-010-007 |
yes | Topic subscription < 50ms (p95) | none |
REQ-002-US-010-008 |
yes | ACK processing < 10ms (p95) | none |
REQ-002-US-010-009 |
yes | Maximum 1GB memory usage | none |
REQ-002-US-010-010 |
yes | Maximum 10GB disk usage | none |
REQ-002-US-010-011 |
yes | Maximum 100 threads | none |
REQ-002-US-010-012 |
yes | Efficient CPU utilization (<50% at peak) | none |
REQ-002-US-010-013 |
yes | Load test with 1000 messages/sec | none |
REQ-002-US-010-014 |
yes | Load test with 5000 messages/sec | none |
REQ-002-US-010-015 |
yes | Load test with 10000 messages/sec | none |
REQ-002-US-010-016 |
yes | Stress test with 100 concurrent agents | none |
REQ-002-US-010-017 |
yes | Stress test with 1000 concurrent agents | none |
REQ-002-US-010-018 |
yes | Spike test (sudden load increase) | none |
REQ-002-US-010-019 |
yes | Soak test (24-hour sustained load) | none |
REQ-002-US-010-020 |
yes | Message batching | none |
REQ-002-US-010-021 |
yes | Connection pooling | none |
REQ-002-US-010-022 |
yes | Zero-copy message transfer | none |
REQ-002-US-010-023 |
yes | Lock-free data structures | none |
REQ-002-US-010-024 |
yes | Asynchronous I/O | none |
REQ-002-FR1-001 |
yes | message_id field (UUID) |
1 |
REQ-002-FR1-002 |
yes | sender field (agent ID) |
1 |
REQ-002-FR1-003 |
yes | recipient field (agent ID or topic) |
1 |
REQ-002-FR1-004 |
yes | timestamp field (ISO 8601) |
2 |
REQ-002-FR1-005 |
yes | priority field (0-4) |
1 |
REQ-002-FR1-006 |
yes | ttl field (seconds) |
1 |
REQ-002-FR1-007 |
yes | correlation_id field (for request-response) |
3 |
REQ-002-FR1-008 |
yes | reply_to field (for responses) |
1 |
REQ-002-FR1-009 |
yes | content_type field (MIME type) |
1 |
REQ-002-FR1-010 |
yes | Validate all required fields present | 4 |
REQ-002-FR1-011 |
yes | Validate field types | 2 |
REQ-002-FR1-012 |
yes | Validate field constraints (e.g., priority range) | 3 |
REQ-002-FR1-013 |
yes | Validate message_id uniqueness | 1 |
REQ-002-FR1-014 |
yes | Reject invalid envelopes | 5 |
REQ-002-FR1-015 |
yes | Parse envelope from JSON | 4 |
REQ-002-FR1-016 |
yes | Serialize envelope to JSON | 4 |
REQ-002-FR1-017 |
yes | Extract routing information | 2 |
REQ-002-FR1-018 |
yes | Extract priority information | 1 |
REQ-002-FR1-019 |
yes | Generate envelope from message | 3 |
REQ-002-FR2-001 |
yes | Plain text payloads | 4 |
REQ-002-FR2-002 |
yes | JSON payloads | 5 |
REQ-002-FR2-003 |
yes | Binary payloads | 4 |
REQ-002-FR2-004 |
yes | Compressed payloads | 7 |
REQ-002-FR2-005 |
yes | Encrypted payloads | none |
REQ-002-FR2-006 |
yes | Validate payload size (max 1MB) | 3 |
REQ-002-FR2-007 |
yes | Validate content_type matches payload | 2 |
REQ-002-FR2-008 |
yes | Validate JSON structure | 1 |
REQ-002-FR2-009 |
yes | Validate character encoding | 1 |
REQ-002-FR2-010 |
yes | Reject oversized payloads | 1 |
REQ-002-FR2-011 |
yes | Parse JSON payloads | 2 |
REQ-002-FR2-012 |
yes | Decompress compressed payloads | 4 |
REQ-002-FR2-013 |
yes | Decrypt encrypted payloads | none |
REQ-002-FR2-014 |
yes | Handle binary data | 2 |
REQ-002-FR2-015 |
yes | Preserve payload integrity | none |
REQ-002-FR3-001 |
yes | Direct routing (agent-to-agent) | 2 |
REQ-002-FR3-002 |
yes | Topic-based routing | 2 |
REQ-002-FR3-003 |
yes | Queue-based routing | none |
REQ-002-FR3-004 |
yes | Broadcast routing | 2 |
REQ-002-FR3-005 |
yes | Multicast routing | 2 |
REQ-002-FR3-006 |
yes | Resolve recipient by agent ID | 3 |
REQ-002-FR3-007 |
yes | Resolve recipients by topic | 2 |
REQ-002-FR3-008 |
yes | Resolve recipients by queue | none |
REQ-002-FR3-009 |
yes | Resolve recipients by filter | 3 |
REQ-002-FR3-010 |
yes | Handle unresolved routes | 3 |
REQ-002-FR3-011 |
yes | Cache resolved routes | 3 |
REQ-002-FR3-012 |
yes | Invalidate cache on changes | 2 |
REQ-002-FR3-013 |
yes | Configure cache size | none |
REQ-002-FR3-014 |
yes | Configure cache TTL | 1 |
REQ-002-FR3-015 |
yes | Cache hit/miss metrics | none |
REQ-002-FR4-001 |
yes | In-memory queue implementation | 2 |
REQ-002-FR4-002 |
yes | Disk-backed queue implementation | 4 |
REQ-002-FR4-003 |
yes | Priority queue implementation | 3 |
REQ-002-FR4-004 |
yes | Bounded queue with overflow handling | 5 |
REQ-002-FR4-005 |
yes | Concurrent queue access | 2 |
REQ-002-FR4-006 |
yes | Enqueue message | 12 |
REQ-002-FR4-007 |
yes | Dequeue message | 10 |
REQ-002-FR4-008 |
yes | Peek message | 6 |
REQ-002-FR4-009 |
yes | Get queue size | 7 |
REQ-002-FR4-010 |
yes | Clear queue | 6 |
REQ-002-FR4-011 |
yes | Iterate queue (for admin) | 2 |
REQ-002-FR4-012 |
yes | Write-ahead log for durability | none |
REQ-002-FR4-013 |
yes | Periodic checkpoints | 11 |
REQ-002-FR4-014 |
yes | Recovery on startup | 6 |
REQ-002-FR4-015 |
yes | Corruption detection | 2 |
REQ-002-FR4-016 |
yes | Repair corrupted queues | none |
REQ-002-FR5-001 |
yes | Store topic definitions | 4 |
REQ-002-FR5-002 |
yes | Store topic subscriptions | 1 |
REQ-002-FR5-003 |
yes | Store topic metadata | 2 |
REQ-002-FR5-004 |
yes | Persist registry to disk | none |
REQ-002-FR5-005 |
yes | Load registry on startup | none |
REQ-002-FR5-006 |
yes | Create topic | 3 |
REQ-002-FR5-007 |
yes | Delete topic | 3 |
REQ-002-FR5-008 |
yes | List topics | 2 |
REQ-002-FR5-009 |
yes | Query topic info | 3 |
REQ-002-FR5-010 |
yes | Subscribe to topic | none |
REQ-002-FR5-011 |
yes | Unsubscribe from topic | 1 |
REQ-002-FR5-012 |
yes | Exact match | none |
REQ-002-FR5-013 |
yes | Wildcard match (*) | 3 |
REQ-002-FR5-014 |
yes | Pattern match (regex) | 1 |
REQ-002-FR5-015 |
yes | Hierarchical match (topic.subtopic) | none |
REQ-002-FR5-016 |
yes | Efficient matching algorithm | none |
REQ-002-FR6-001 |
yes | Store subscriptions in memory | 2 |
REQ-002-FR6-002 |
yes | Persist subscriptions to disk | none |
REQ-002-FR6-003 |
yes | Index subscriptions by topic | none |
REQ-002-FR6-004 |
yes | Index subscriptions by agent | 1 |
REQ-002-FR6-005 |
yes | Handle large subscription counts | none |
REQ-002-FR6-006 |
yes | Add subscription | 2 |
REQ-002-FR6-007 |
yes | Remove subscription | 5 |
REQ-002-FR6-008 |
yes | List subscriptions by agent | 2 |
REQ-002-FR6-009 |
yes | List subscriptions by topic | 1 |
REQ-002-FR6-010 |
yes | Update subscription metadata | none |
REQ-002-FR6-011 |
yes | Match message to subscriptions | 3 |
REQ-002-FR6-012 |
yes | Return all matching subscribers | 3 |
REQ-002-FR6-013 |
yes | Filter by agent status (active) | none |
REQ-002-FR6-014 |
yes | Cache frequent matches | none |
REQ-002-FR6-015 |
yes | Optimize for common patterns | none |
REQ-002-FR7-001 |
yes | Generate receipt on message receipt | 1 |
REQ-002-FR7-002 |
yes | Generate receipt on message delivery | 2 |
REQ-002-FR7-003 |
yes | Generate receipt on message failure | 2 |
REQ-002-FR7-004 |
yes | Include delivery timestamp | 1 |
REQ-002-FR7-005 |
yes | Include failure reason (if applicable) | 1 |
REQ-002-FR7-006 |
yes | Store receipts in memory | 4 |
REQ-002-FR7-007 |
yes | Persist receipts to disk | 5 |
REQ-002-FR7-008 |
yes | Index by message ID | 3 |
REQ-002-FR7-009 |
yes | Index by recipient | none |
REQ-002-FR7-010 |
yes | Expire old receipts | 3 |
REQ-002-FR7-011 |
yes | Query receipt by message ID | 3 |
REQ-002-FR7-012 |
yes | Query receipts by time range | 2 |
REQ-002-FR7-013 |
yes | Query receipts by sender | 1 |
REQ-002-FR7-014 |
yes | Query receipts by recipient | 1 |
REQ-002-FR7-015 |
yes | Export receipts for audit | none |
REQ-002-FR8-001 |
yes | Connection errors | none |
REQ-002-FR8-002 |
yes | Timeout errors | none |
REQ-002-FR8-003 |
yes | Validation errors | none |
REQ-002-FR8-004 |
yes | Authorization errors | none |
REQ-002-FR8-005 |
yes | System errors (OOM, disk full) | none |
REQ-002-FR8-006 |
yes | Generate error message | 3 |
REQ-002-FR8-007 |
yes | Include error code | 2 |
REQ-002-FR8-008 |
yes | Include error description | 3 |
REQ-002-FR8-009 |
yes | Include stack trace (if debug) | 3 |
REQ-002-FR8-010 |
yes | Send error to sender | none |
REQ-002-FR8-011 |
yes | Log all errors | 1 |
REQ-002-FR8-012 |
yes | Include context information | 3 |
REQ-002-FR8-013 |
yes | Include timestamp | none |
REQ-002-FR8-014 |
yes | Rotate log files | none |
REQ-002-FR8-015 |
yes | Export logs for analysis | none |
REQ-002-FR8-016 |
yes | Retry transient errors | 3 |
REQ-002-FR8-017 |
yes | Fallback for permanent errors | 3 |
REQ-002-FR8-018 |
yes | Alert on critical errors | none |
REQ-002-FR8-019 |
yes | Graceful degradation | 2 |
REQ-002-FR8-020 |
yes | Circuit breaker pattern | 16 |
REQ-002-FR9-001 |
yes | Collect core metrics (see US-008) | 3 |
REQ-002-FR9-002 |
yes | Aggregate metrics by time window | none |
REQ-002-FR9-003 |
yes | Calculate percentiles (p50, p95, p99) | none |
REQ-002-FR9-004 |
yes | Track metric trends | none |
REQ-002-FR9-005 |
yes | Alert on anomalies | none |
REQ-002-FR9-006 |
yes | /metrics endpoint (Prometheus format) |
none |
REQ-002-FR9-007 |
yes | /health endpoint (health check) |
10 |
REQ-002-FR9-008 |
yes | /stats endpoint (JSON stats) |
none |
REQ-002-FR9-009 |
yes | Real-time metrics WebSocket | none |
REQ-002-FR9-010 |
yes | Historical metrics API | none |
REQ-002-FR9-011 |
yes | Structured logging (JSON) | none |
REQ-002-FR9-012 |
yes | Log levels (DEBUG, INFO, WARN, ERROR) | none |
REQ-002-FR9-013 |
yes | Log filtering | none |
REQ-002-FR9-014 |
yes | Log aggregation | none |
REQ-002-FR9-015 |
yes | Log search | none |
REQ-002-FR10-001 |
yes | List all agents | 6 |
REQ-002-FR10-002 |
yes | List all queues | none |
REQ-002-FR10-003 |
yes | List all topics | none |
REQ-002-FR10-004 |
yes | View agent details | none |
REQ-002-FR10-005 |
yes | View queue details | none |
REQ-002-FR10-006 |
yes | View topic details | none |
REQ-002-FR10-007 |
yes | Purge queue | none |
REQ-002-FR10-008 |
yes | Delete topic | none |
REQ-002-FR10-009 |
yes | Disconnect agent | 4 |
REQ-002-FR10-010 |
yes | Replay messages | 1 |
REQ-002-FR10-011 |
yes | Export configuration | 4 |
REQ-002-FR10-012 |
yes | Import configuration | 4 |
REQ-002-FR10-013 |
yes | REST API for admin operations | none |
REQ-002-FR10-014 |
yes | Authentication required | none |
REQ-002-FR10-015 |
yes | Authorization by role | 1 |
REQ-002-FR10-016 |
yes | Audit log all admin actions | none |
REQ-002-FR10-017 |
yes | Rate limit admin requests | none |
REQ-002-FR11-001 |
yes | Load from postman.yaml | 2 |
REQ-002-FR11-002 |
yes | Load from environment variables | 4 |
REQ-002-FR11-003 |
yes | Load from CLI arguments | none |
REQ-002-FR11-004 |
yes | Merge configurations | 2 |
REQ-002-FR11-005 |
yes | Validate loaded configuration | 4 |
REQ-002-FR11-006 |
yes | Define YAML schema | none |
REQ-002-FR11-007 |
yes | Validate against schema | none |
REQ-002-FR11-008 |
yes | Provide schema documentation | none |
REQ-002-FR11-009 |
yes | Default values for all settings | 2 |
REQ-002-FR11-010 |
yes | Required vs optional settings | none |
REQ-002-FR11-011 |
yes | Watch configuration file | 3 |
REQ-002-FR11-012 |
yes | Detect changes | 2 |
REQ-002-FR11-013 |
yes | Validate new configuration | none |
REQ-002-FR11-014 |
yes | Apply changes atomically | none |
REQ-002-FR11-015 |
yes | Rollback on failure | none |
REQ-002-FR11-016 |
yes | Postman facts stored in Prolog KB | none |
REQ-002-FR11-017 |
yes | Query message routing via Prolog | none |
REQ-002-FR11-018 |
yes | Query subscription rules via Prolog | none |
REQ-002-FR11-019 |
yes | Update Postman state in Prolog | none |
REQ-002-FR11-020 |
yes | Prolog-based message filtering | none |
REQ-002-FR11-021 |
yes | Agent registration with Postman | 2 |
REQ-002-FR11-022 |
yes | Agent deregistration | none |
REQ-002-FR11-023 |
yes | Agent message send/receive | none |
REQ-002-FR11-024 |
yes | Agent status tracking | none |
REQ-002-FR11-025 |
yes | Agent capability discovery | none |
REQ-002-FR11-026 |
yes | Session-based message isolation | none |
REQ-002-FR11-027 |
yes | Per-session queues | none |
REQ-002-FR11-028 |
yes | Per-session topics | none |
REQ-002-FR11-029 |
yes | Session cleanup on end | none |
REQ-002-FR11-030 |
yes | Session message history | none |
REQ-002-FR11-031 |
yes | Installation guide | none |
REQ-002-FR11-032 |
yes | Configuration guide | none |
REQ-002-FR11-033 |
yes | User manual | none |
REQ-002-FR11-034 |
yes | API reference | none |
REQ-002-FR11-035 |
yes | Examples and tutorials | none |
REQ-002-FR11-036 |
yes | Architecture overview | none |
REQ-002-FR11-037 |
yes | Component design docs | none |
REQ-002-FR11-038 |
yes | API documentation | none |
REQ-002-FR11-039 |
yes | Testing guide | none |
REQ-002-FR11-040 |
yes | Contribution guide | none |
REQ-002-FR11-041 |
yes | Deployment guide | none |
REQ-002-FR11-042 |
yes | Monitoring guide | none |
REQ-002-FR11-043 |
yes | Troubleshooting guide | none |
REQ-002-FR11-044 |
yes | Performance tuning guide | none |
REQ-002-FR11-045 |
yes | Backup/restore procedures | none |
Code that traces this spec
Checklists
checklists/README.md
Postman Integration Checklists
Project: Orchestrator-AI Postman Integration (Spec 002) Version: 1.0.0 Last Updated: 2026-01-13
Overview
This directory contains comprehensive checklists for implementing and verifying the Postman Integration component of the Orchestrator-AI system. The checklists cover requirements, constitutional compliance, and testing.
Checklist Files
1. requirements.md
Purpose: Track implementation of all functional requirements and user stories Total Checkboxes: 457 Coverage:
- 10 User Stories (US-001 through US-010)
- 11 Functional Requirements (FR-001 through FR-011)
- Integration requirements
- Documentation requirements
Key Sections:
- Agent Message Delivery (US-001)
- Topic-Based Subscriptions (US-002)
- Message Queue Management (US-003)
- Message Priority Handling (US-004)
- Message Filtering and Routing (US-005)
- Message Persistence and Replay (US-006)
- Delivery Guarantees and Acknowledgments (US-007)
- Monitoring and Metrics (US-008)
- Configuration Management (US-009)
- Performance and Scalability (US-010)
2. constitutional.md
Purpose: Ensure compliance with all 11 articles of the Orchestrator-AI Constitution Total Checkboxes: 355 Coverage:
- Agent Autonomy (Article I)
- Prolog as Single Source of Truth (Article II)
- Session Isolation (Article III)
- Hooks for Extensibility (Article IV)
- Minimal Core, Maximal Extension (Article V)
- Python 3.11+ Implementation (Article VI)
- Testing Without Mocks (Article VII)
- Metrics and Observability (Article VIII)
- Graceful Degradation (Article IX)
- Version Compatibility (Article X)
- Documentation First (Article XI)
Key Sections:
- Agent autonomy verification
- Prolog fact schemas and queries
- Session isolation enforcement
- Hook system implementation
- Code quality standards
- Testing philosophy
- Metrics export formats
- Graceful degradation scenarios
3. testing.md
Purpose: Comprehensive testing plan including unit, integration, contract, and performance tests Total Checkboxes: 490 Coverage:
- Unit tests (160+ tests)
- Integration tests (120+ tests)
- Contract tests (55 tests)
- Performance tests (90+ tests)
- Compliance tests (11 tests)
- Failure scenario tests (54+ tests)
Key Sections:
- Message envelope tests
- Queue implementation tests
- Topic management tests
- Routing engine tests
- Throughput tests (5000 msg/sec target)
- Latency tests (<100ms p95 target)
- Scalability tests
- Stress and endurance tests
Total Checklist Summary
| File | Checkboxes | Lines | Size |
|---|---|---|---|
| requirements.md | 457 | 788 | 20K |
| constitutional.md | 355 | 590 | 18K |
| testing.md | 490 | 645 | 22K |
| TOTAL | 1,302 | 2,023 | 60K |
Usage Guidelines
For Implementation
- Start with
requirements.mdto understand what needs to be built - Check
constitutional.mdto ensure compliance with architectural principles - Use
testing.mdto guide test development and verification
For Reviews
- Verify all checkboxes in relevant sections are addressed
- Ensure compliance items in
constitutional.mdare satisfied - Confirm test coverage meets requirements in
testing.md
For Project Tracking
- Update checkboxes as work progresses
- Track completion percentage in each file
- Generate progress reports from checkbox counts
- Identify blockers and dependencies
Checkbox Status
[ ]- Not started[x]- Completed[~]- In progress (optional notation)[!]- Blocked (optional notation)
Key Metrics and Targets
Performance Targets
- Throughput: 5,000+ messages/second sustained
- Latency: <100ms (p95), <200ms (p99)
- Scalability: 1,000 concurrent agents, 1,000 queues, 1,000 topics
- Memory: <1GB at target throughput
- CPU: <50% utilization at peak load
Quality Targets
- Test Coverage: >90% code coverage
- Constitutional Compliance: 100% (all articles)
- Documentation: Complete before implementation
- No Mocking: 100% real implementations in tests
Functional Completeness
- User Stories: 10/10 implemented
- Functional Requirements: 11/11 implemented
- Integration Points: Prolog, Agents, Sessions
- Extension Hooks: 20+ hook points
Dependencies
Core Dependencies
- Python 3.11+
- Prolog KB (SWI-Prolog)
- Agent Framework
- Session Management System
Testing Dependencies
- pytest (unit testing)
- Real Prolog instance (no mocking)
- Real file system (temp directories)
- Performance testing tools
Documentation Dependencies
- Markdown
- Mermaid (diagrams)
- API documentation tools
Review Process
Checklist Review
- Initial Review: Architecture team reviews all checklists
- Implementation Review: Developers review during implementation
- Testing Review: QA team reviews test checklists
- Final Review: All stakeholders review before release
Sign-off Requirements
- [ ] Architecture approval (constitutional compliance)
- [ ] Development approval (requirements completeness)
- [ ] QA approval (testing adequacy)
- [ ] Documentation approval (docs completeness)
Maintenance
Updates
- Update checklists when requirements change
- Add new tests as edge cases are discovered
- Revise compliance checks as constitution evolves
- Keep metrics aligned with performance targets
Version Control
- Checklists are versioned with the spec
- Track changes in git history
- Document major revisions in commit messages
- Tag checklist versions with releases
References
- Main Specification:
/specs/002-postman/002-postman-integration.md - Constitution:
/specs/orchestrator-constitution.md - Architecture:
/docs/architecture/postman-architecture.md(TBD) - API Reference:
/docs/api/postman-api.md(TBD)
Quick Stats
Total Checkboxes: 1,302
Total Lines: 2,023
Total Size: 60K
Estimated Work: 3-6 months (with team)
Critical Path: Performance + Prolog Integration
Contact
For questions or updates to these checklists:
- Create an issue in the project tracker
- Contact the architecture team
- Refer to the main specification document
checklists/SUMMARY.txt
================================================================================
POSTMAN INTEGRATION CHECKLISTS - CREATION SUMMARY
================================================================================
Project: Orchestrator-AI Postman Integration (Spec 002)
Created: 2026-01-13
Location: /home/mohamed/workspace/research/orchestrator-ai/specs/002-postman/checklists/
================================================================================
FILES CREATED
================================================================================
1. requirements.md
- Purpose: Track all functional requirements and user stories
- Size: 20K
- Lines: 788
- Checkboxes: 457
- Sections:
* 10 User Stories (US-001 to US-010)
* 11 Functional Requirements (FR-001 to FR-011)
* Integration requirements
* Documentation requirements
2. constitutional.md
- Purpose: Ensure compliance with Orchestrator-AI Constitution
- Size: 18K
- Lines: 590
- Checkboxes: 355
- Sections:
* Article I: Agent Autonomy
* Article II: Prolog as Single Source of Truth
* Article III: Session Isolation
* Article IV: Hooks for Extensibility
* Article V: Minimal Core, Maximal Extension
* Article VI: Python 3.11+ Implementation
* Article VII: Testing Without Mocks
* Article VIII: Metrics and Observability
* Article IX: Graceful Degradation
* Article X: Version Compatibility
* Article XI: Documentation First
3. testing.md
- Purpose: Comprehensive testing plan and verification
- Size: 22K
- Lines: 645
- Checkboxes: 490
- Sections:
* Unit Tests (160+ tests)
* Integration Tests (120+ tests)
* Contract Tests (55 tests)
* Performance Tests (90+ tests)
* Compliance Tests (11 tests)
* Failure Scenario Tests (54+ tests)
4. README.md
- Purpose: Overview and usage guide for all checklists
- Size: 6.2K
- Lines: 196
- Checkboxes: 4 (sign-off requirements)
- Sections:
* File descriptions
* Usage guidelines
* Key metrics and targets
* Review process
* Maintenance procedures
================================================================================
STATISTICS
================================================================================
Total Files: 4
Total Size: 68K
Total Lines: 2,219
Total Checkboxes: 1,306 (including README)
Breakdown by Category:
- Requirements Tracking: 457 checkboxes (35%)
- Constitutional Compliance: 355 checkboxes (27%)
- Testing & Verification: 490 checkboxes (38%)
================================================================================
KEY COVERAGE AREAS
================================================================================
USER STORIES (10 total):
✓ US-001: Agent Message Delivery via Postman
✓ US-002: Topic-Based Message Subscription
✓ US-003: Message Queue Management
✓ US-004: Message Priority Handling
✓ US-005: Message Filtering and Routing
✓ US-006: Message Persistence and Replay
✓ US-007: Delivery Guarantees and Acknowledgments
✓ US-008: Monitoring and Metrics
✓ US-009: Configuration Management
✓ US-010: Performance and Scalability
FUNCTIONAL REQUIREMENTS (11 total):
✓ FR-001: Message Envelope Structure
✓ FR-002: Message Payload Handling
✓ FR-003: Routing Engine
✓ FR-004: Queue Implementation
✓ FR-005: Topic Management
✓ FR-006: Subscription Manager
✓ FR-007: Delivery Receipts
✓ FR-008: Error Handling
✓ FR-009: Monitoring Interface
✓ FR-010: Administrative Interface
✓ FR-011: Configuration System
CONSTITUTIONAL ARTICLES (11 total):
✓ Article I: Agent Autonomy
✓ Article II: Prolog as Single Source of Truth
✓ Article III: Session Isolation
✓ Article IV: Hooks for Extensibility
✓ Article V: Minimal Core, Maximal Extension
✓ Article VI: Python 3.11+ Implementation
✓ Article VII: Testing Without Mocks
✓ Article VIII: Metrics and Observability
✓ Article IX: Graceful Degradation
✓ Article X: Version Compatibility
✓ Article XI: Documentation First
TESTING CATEGORIES:
✓ Unit Tests (message, queue, topic, routing, etc.)
✓ Integration Tests (Prolog, agents, sessions, hooks)
✓ Contract Tests (API, schema, backward compatibility)
✓ Performance Tests (throughput, latency, scalability)
✓ Compliance Tests (constitutional, security, integrity)
✓ Failure Scenario Tests (network, system, component failures)
================================================================================
PERFORMANCE TARGETS
================================================================================
Throughput: 5,000+ messages/second sustained
Latency (p95): <100ms
Latency (p99): <200ms
Scalability: 1,000 concurrent agents, 1,000 queues, 1,000 topics
Memory Usage: <1GB at target throughput
CPU Usage: <50% utilization at peak load
Test Coverage: >90% code coverage
Constitutional Compliance: 100%
================================================================================
ESTIMATED EFFORT
================================================================================
Based on 1,302 checkboxes with varying complexity:
- Requirements Implementation: 8-12 weeks
- Constitutional Compliance: 4-6 weeks
- Testing & Verification: 6-8 weeks
- Documentation: 2-4 weeks
Total Estimated Duration: 3-6 months (with team of 3-5 developers)
Critical Path:
1. Core message delivery (US-001, FR-001, FR-002, FR-003)
2. Queue and topic infrastructure (US-002, US-003, FR-004, FR-005)
3. Prolog integration (Article II)
4. Performance optimization (US-010)
5. Comprehensive testing (all test categories)
================================================================================
NEXT STEPS
================================================================================
1. Review checklists with architecture team
2. Prioritize requirements based on dependencies
3. Create detailed implementation plan
4. Assign work to development team
5. Begin implementation with daily checklist updates
6. Conduct regular reviews against checklists
7. Track completion metrics
8. Verify all compliance requirements
================================================================================
USAGE INSTRUCTIONS
================================================================================
For Developers:
- Check off items as you complete them
- Update progress regularly
- Reference requirements during implementation
- Verify constitutional compliance for each component
- Write tests according to testing checklist
For Reviewers:
- Verify all relevant checkboxes are addressed
- Ensure code matches requirements
- Confirm constitutional compliance
- Validate test coverage
For Project Managers:
- Track completion percentage
- Identify blockers
- Monitor progress against timeline
- Generate status reports
================================================================================
END OF SUMMARY
================================================================================
checklists/constitutional.md
Constitutional Compliance Checklist - Postman Integration
Project: Orchestrator-AI Postman Integration Version: 1.0.0 Last Updated: 2026-01-13 Reference: orchestrator-constitution.md
Overview
This checklist ensures the Postman Integration complies with all 11 articles of the Orchestrator-AI Constitution. Every component, interface, and implementation must adhere to these constitutional principles.
Total Items: 180+ checkboxes
Article I: Agent Autonomy
Principle: Agents are autonomous entities with explicit capabilities, constraints, and lifecycle management.
Agent Registration (Postman Context)
- [ ] Postman respects agent autonomy in message handling
- [ ] Agents explicitly subscribe to topics (no forced subscriptions)
- [ ] Agents explicitly create queues (no automatic queues)
- [ ] Agents control their message delivery preferences
- [ ] Agents can refuse messages based on their policies
Agent Capabilities Declaration
- [ ] Agents declare message handling capabilities
- [ ] Agents declare supported message types
- [ ] Agents declare maximum message size they accept
- [ ] Agents declare maximum queue depth
- [ ] Agents declare priority preferences
Agent Lifecycle Management
- [ ] Postman tracks agent connection state
- [ ] Postman handles agent disconnection gracefully
- [ ] Postman cleans up agent resources on termination
- [ ] Postman notifies subscribers of agent state changes
- [ ] Postman archives agent message history
Agent State Transitions
- [ ] Document CONNECTED state behavior
- [ ] Document DISCONNECTED state behavior
- [ ] Document IDLE state behavior
- [ ] Document BUSY state behavior
- [ ] Document TERMINATED state behavior
- [ ] Validate state transitions
- [ ] Prevent invalid state changes
Agent Constraints
- [ ] Enforce agent rate limits
- [ ] Enforce agent message size limits
- [ ] Enforce agent queue size limits
- [ ] Reject messages exceeding agent constraints
- [ ] Log constraint violations
Article II: Prolog as Single Source of Truth
Principle: All authoritative knowledge resides in the Prolog knowledge base.
Postman State in Prolog
- [ ] Store active agents in Prolog KB
- [ ] Store active queues in Prolog KB
- [ ] Store active topics in Prolog KB
- [ ] Store active subscriptions in Prolog KB
- [ ] Store routing rules in Prolog KB
- [ ] Store message receipts in Prolog KB (recent only)
- [ ] Store delivery statistics in Prolog KB
Prolog Fact Schemas
- [ ] Define
postman_agent(AgentID, Status, Capabilities)fact - [ ] Define
postman_queue(QueueName, Type, Options)fact - [ ] Define
postman_topic(TopicName, Subscribers)fact - [ ] Define
postman_subscription(AgentID, Topic, Pattern)fact - [ ] Define
postman_route(Pattern, Destination, Priority)fact - [ ] Define
postman_receipt(MessageID, Status, Timestamp)fact - [ ] Define
postman_stats(Metric, Value, Timestamp)fact
Prolog Query Integration
- [ ] Query agent status from Prolog
- [ ] Query subscription list from Prolog
- [ ] Query routing rules from Prolog
- [ ] Query queue configuration from Prolog
- [ ] Query topic subscribers from Prolog
- [ ] Cache Prolog query results appropriately
- [ ] Invalidate cache on Prolog updates
Prolog Update Protocol
- [ ] Update Prolog facts on agent connect/disconnect
- [ ] Update Prolog facts on subscription changes
- [ ] Update Prolog facts on queue operations
- [ ] Update Prolog facts on topic changes
- [ ] Update Prolog facts on message delivery
- [ ] Batch Prolog updates for performance
- [ ] Handle Prolog update failures gracefully
Prolog Synchronization
- [ ] Periodically sync Postman state to Prolog
- [ ] Detect Prolog state divergence
- [ ] Reconcile state conflicts (Prolog wins)
- [ ] Log synchronization events
- [ ] Alert on persistent sync failures
Article III: Session Isolation
Principle: Sessions are isolated execution contexts with clear boundaries.
Session-Based Messaging
- [ ] Associate messages with session IDs
- [ ] Isolate message queues per session
- [ ] Isolate topic subscriptions per session
- [ ] Prevent cross-session message delivery
- [ ] Validate session ID on all operations
Session Queue Management
- [ ] Create session-specific queues
- [ ] Name queues with session prefix (session-{id}-{name})
- [ ] Destroy session queues on session end
- [ ] Prevent access to other session queues
- [ ] Log session queue operations
Session Topic Management
- [ ] Create session-specific topics
- [ ] Name topics with session prefix (session-{id}-{topic})
- [ ] Unsubscribe all session subscribers on end
- [ ] Prevent subscription to other session topics
- [ ] Log session topic operations
Session Message Routing
- [ ] Route messages within session boundaries
- [ ] Reject cross-session message attempts
- [ ] Support global topics (if explicitly configured)
- [ ] Support shared queues (if explicitly configured)
- [ ] Log routing decisions
Session Cleanup
- [ ] Purge session queues on end
- [ ] Remove session subscriptions on end
- [ ] Archive session messages
- [ ] Delete session routing rules
- [ ] Release session resources
Session Statistics
- [ ] Track messages per session
- [ ] Track queue depth per session
- [ ] Track topic activity per session
- [ ] Export session metrics
- [ ] Store session stats in Prolog
Article IV: Hooks for Extensibility
Principle: Behavior is customizable through well-defined hook points.
Message Lifecycle Hooks
- [ ]
pre_sendhook (before message queuing) - [ ]
post_sendhook (after message queuing) - [ ]
pre_deliverhook (before message delivery) - [ ]
post_deliverhook (after message delivery) - [ ]
on_delivery_failurehook (on failed delivery) - [ ]
on_message_expirehook (on TTL expiration)
Subscription Lifecycle Hooks
- [ ]
pre_subscribehook (before subscription) - [ ]
post_subscribehook (after subscription) - [ ]
pre_unsubscribehook (before unsubscribe) - [ ]
post_unsubscribehook (after unsubscribe) - [ ]
on_topic_matchhook (on topic message delivery)
Queue Lifecycle Hooks
- [ ]
pre_enqueuehook (before message enqueue) - [ ]
post_enqueuehook (after message enqueue) - [ ]
pre_dequeuehook (before message dequeue) - [ ]
post_dequeuehook (after message dequeue) - [ ]
on_queue_overflowhook (on queue full) - [ ]
on_queue_underflowhook (on queue empty)
Routing Hooks
- [ ]
pre_routehook (before routing decision) - [ ]
post_routehook (after routing decision) - [ ]
on_route_failurehook (no route found) - [ ]
on_multi_routehook (multiple routes found)
Hook Implementation
- [ ] Define hook interface (Python protocols)
- [ ] Register hooks dynamically
- [ ] Execute hooks in registration order
- [ ] Handle hook exceptions gracefully
- [ ] Log hook execution
- [ ] Support asynchronous hooks
- [ ] Timeout hook execution (max 100ms)
Hook Configuration
- [ ] Configure hooks via YAML
- [ ] Enable/disable hooks dynamically
- [ ] Configure hook priority
- [ ] Configure hook timeout
- [ ] Validate hook configuration
Article V: Minimal Core, Maximal Extension
Principle: Core is minimal and stable; extensions add functionality.
Core Postman Functionality
- [ ] Message envelope handling (core)
- [ ] Direct agent-to-agent delivery (core)
- [ ] Basic queue (FIFO) implementation (core)
- [ ] Delivery receipts (core)
- [ ] Error handling (core)
Extension Modules
- [ ] Topic subscription (extension)
- [ ] Priority queue (extension)
- [ ] Message filtering (extension)
- [ ] Message persistence (extension)
- [ ] Message replay (extension)
- [ ] Advanced routing (extension)
- [ ] Monitoring/metrics (extension)
Extension Interface
- [ ] Define PostmanExtension base class
- [ ] Extension registration API
- [ ] Extension lifecycle management
- [ ] Extension dependency resolution
- [ ] Extension configuration
- [ ] Extension isolation (no core modification)
Core Stability
- [ ] Core API is versioned
- [ ] Core API is backward compatible
- [ ] Core behavior is well-documented
- [ ] Core has comprehensive tests
- [ ] Core changes require review
Extension Guidelines
- [ ] Extensions use hooks only
- [ ] Extensions don't modify core state directly
- [ ] Extensions are independently testable
- [ ] Extensions declare dependencies
- [ ] Extensions provide documentation
Article VI: Python 3.11+ Implementation
Principle: Implementation uses modern Python features and best practices.
Python Version Compliance
- [ ] Require Python 3.11 or higher
- [ ] Use Python 3.11+ features (match/case, etc.)
- [ ] Use type hints throughout
- [ ] Use async/await for I/O operations
- [ ] Use dataclasses for data structures
Type Annotations
- [ ] Annotate all function signatures
- [ ] Annotate all class attributes
- [ ] Annotate all module variables
- [ ] Use generic types where appropriate
- [ ] Use Protocol for interfaces
Modern Python Features
- [ ] Use
match/casefor message routing - [ ] Use
asynciofor concurrency - [ ] Use
pathlibfor file operations - [ ] Use
dataclassesfor message envelopes - [ ] Use
typing.Protocolfor interfaces
Code Quality Standards
- [ ] Pass
mypytype checking - [ ] Pass
blackformatting - [ ] Pass
flake8linting - [ ] Pass
pylintchecks (score > 9.0) - [ ] Follow PEP 8 style guide
Package Management
- [ ] Use
uvfor dependencies (no pip) - [ ] Pin dependency versions
- [ ] Document dependencies in pyproject.toml
- [ ] Use virtual environments
- [ ] Test with isolated environments
Article VII: Testing Without Mocks
Principle: Tests use real implementations, not mocks.
Real Component Testing
- [ ] Test with real queues (no mock queues)
- [ ] Test with real topics (no mock topics)
- [ ] Test with real agents (no mock agents)
- [ ] Test with real Prolog KB (no mock KB)
- [ ] Test with real file system (temp dirs)
Integration Tests
- [ ] Test Postman with Prolog integration
- [ ] Test Postman with Agent integration
- [ ] Test Postman with Session integration
- [ ] Test multi-component workflows
- [ ] Test failure scenarios
Test Data
- [ ] Use real message payloads
- [ ] Use realistic agent configurations
- [ ] Use realistic queue sizes
- [ ] Use realistic topic hierarchies
- [ ] Generate test data programmatically
Test Environment
- [ ] Create isolated test environments
- [ ] Clean up after each test
- [ ] Use temporary directories
- [ ] Use separate Prolog KB for tests
- [ ] Reset state between tests
No Mocking Policy
- [ ] No
unittest.mockusage - [ ] No
MagicMockusage - [ ] No
patchdecorators - [ ] Use test doubles (real implementations)
- [ ] Use in-memory alternatives for speed
Article VIII: Metrics and Observability
Principle: All components expose metrics for monitoring and debugging.
Core Metrics
- [ ] Messages sent counter
- [ ] Messages received counter
- [ ] Messages queued gauge
- [ ] Messages delivered counter
- [ ] Messages failed counter
- [ ] Active agents gauge
- [ ] Active queues gauge
- [ ] Active topics gauge
Latency Metrics
- [ ] Message send latency histogram
- [ ] Message delivery latency histogram
- [ ] Queue enqueue latency histogram
- [ ] Queue dequeue latency histogram
- [ ] Topic subscription latency histogram
- [ ] Route resolution latency histogram
Throughput Metrics
- [ ] Messages per second (sent)
- [ ] Messages per second (delivered)
- [ ] Bytes per second (sent)
- [ ] Bytes per second (delivered)
- [ ] Queue operations per second
- [ ] Topic operations per second
Error Metrics
- [ ] Delivery failures counter
- [ ] Timeout errors counter
- [ ] Validation errors counter
- [ ] System errors counter
- [ ] Error rate per minute
Resource Metrics
- [ ] Memory usage gauge
- [ ] CPU usage gauge
- [ ] Disk usage gauge
- [ ] Network bandwidth gauge
- [ ] Thread pool utilization gauge
Metric Export
- [ ] Prometheus
/metricsendpoint - [ ] JSON metrics API
- [ ] Real-time metrics WebSocket
- [ ] Metric aggregation (1min, 5min, 1hr)
- [ ] Metric retention (7 days)
Structured Logging
- [ ] JSON log format
- [ ] Include correlation IDs
- [ ] Include session IDs
- [ ] Include agent IDs
- [ ] Include timestamps (ISO 8601)
- [ ] Include log levels
- [ ] Include context information
Distributed Tracing
- [ ] Generate trace IDs for messages
- [ ] Propagate trace IDs through hops
- [ ] Export traces in OpenTelemetry format
- [ ] Include span information
- [ ] Link related traces
Article IX: Graceful Degradation
Principle: System degrades gracefully under failure conditions.
Failure Scenarios
- [ ] Handle agent disconnection gracefully
- [ ] Handle Prolog KB unavailability
- [ ] Handle disk full conditions
- [ ] Handle memory exhaustion
- [ ] Handle network partitions
- [ ] Handle CPU overload
Degraded Mode Operations
- [ ] Operate without Prolog (cache only)
- [ ] Operate with limited disk (memory only)
- [ ] Operate with limited memory (disk fallback)
- [ ] Operate without persistence (ephemeral)
- [ ] Shed load under high pressure
Circuit Breaker Pattern
- [ ] Implement circuit breaker for Prolog calls
- [ ] Implement circuit breaker for disk writes
- [ ] Implement circuit breaker for agent connections
- [ ] Track circuit breaker state
- [ ] Auto-recover from open circuits
Backpressure Handling
- [ ] Apply backpressure on queue full
- [ ] Apply backpressure on high memory
- [ ] Apply backpressure on slow consumers
- [ ] Signal backpressure to senders
- [ ] Throttle message acceptance
Fallback Behaviors
- [ ] Queue to disk if memory full
- [ ] Drop low-priority messages if overloaded
- [ ] Reject new subscriptions if overloaded
- [ ] Return cached data if Prolog unavailable
- [ ] Log all fallback activations
Recovery Procedures
- [ ] Detect system recovery
- [ ] Resume normal operations
- [ ] Process queued messages
- [ ] Sync state to Prolog
- [ ] Notify agents of recovery
Article X: Version Compatibility
Principle: Interfaces are versioned and backward compatible.
API Versioning
- [ ] Version Postman API (v1, v2, etc.)
- [ ] Include version in message envelope
- [ ] Support multiple API versions simultaneously
- [ ] Deprecate old versions gradually (6 months)
- [ ] Document version differences
Message Format Versioning
- [ ] Version message envelope schema
- [ ] Support multiple envelope versions
- [ ] Convert between envelope versions
- [ ] Validate envelope version
- [ ] Reject unsupported versions
Protocol Versioning
- [ ] Version Postman protocol
- [ ] Negotiate protocol version on connect
- [ ] Support protocol version fallback
- [ ] Document protocol versions
- [ ] Test cross-version compatibility
Backward Compatibility
- [ ] Support v1 message format in v2
- [ ] Support v1 API in v2 server
- [ ] Maintain old configuration keys
- [ ] Provide migration tools
- [ ] Document breaking changes
Deprecation Policy
- [ ] Announce deprecations in release notes
- [ ] Log warnings for deprecated features
- [ ] Provide 6-month deprecation period
- [ ] Document migration path
- [ ] Remove after deprecation period
Article XI: Documentation First
Principle: Documentation is written before implementation.
Design Documentation
- [ ] Postman architecture document
- [ ] Component interaction diagrams
- [ ] Data flow diagrams
- [ ] State machine diagrams
- [ ] Sequence diagrams
API Documentation
- [ ] REST API reference
- [ ] Message envelope schema
- [ ] Queue API documentation
- [ ] Topic API documentation
- [ ] Subscription API documentation
- [ ] Hook API documentation
User Documentation
- [ ] Installation guide
- [ ] Configuration guide
- [ ] Quick start guide
- [ ] User manual
- [ ] FAQ document
- [ ] Troubleshooting guide
Developer Documentation
- [ ] Development setup guide
- [ ] Testing guide
- [ ] Extension development guide
- [ ] Hook development guide
- [ ] Code style guide
- [ ] Contribution guidelines
Operational Documentation
- [ ] Deployment guide
- [ ] Monitoring guide
- [ ] Performance tuning guide
- [ ] Backup/restore procedures
- [ ] Disaster recovery plan
- [ ] Incident response procedures
Documentation Standards
- [ ] Use Markdown format
- [ ] Include code examples
- [ ] Include diagrams (Mermaid)
- [ ] Version documentation with code
- [ ] Review documentation before merge
- [ ] Update documentation with changes
Compliance Verification
Automated Checks
- [ ] Run constitutional compliance tests
- [ ] Check Prolog integration
- [ ] Check session isolation
- [ ] Check hook system
- [ ] Check metrics export
- [ ] Check error handling
Manual Reviews
- [ ] Code review for constitutional compliance
- [ ] Architecture review for principles
- [ ] Documentation review for completeness
- [ ] Security review for isolation
- [ ] Performance review for scalability
Continuous Compliance
- [ ] Add constitutional checks to CI/CD
- [ ] Block merges on compliance failures
- [ ] Generate compliance report
- [ ] Track compliance metrics
- [ ] Review compliance quarterly
Completion Summary
Total Checkboxes: 180+ Completed: 0 In Progress: 0 Not Started: 180+
Progress: 0%
Notes
- Constitutional compliance is mandatory for all components
- Violations must be documented and justified
- All exceptions require architecture review approval
- Compliance is verified in CI/CD pipeline
checklists/requirements.md
The requirement list above comes from this file; 457 of 457 checkbox items are ticked.
checklists/testing.md
Testing Checklist - Postman Integration
Project: Orchestrator-AI Postman Integration Version: 1.0.0 Last Updated: 2026-01-13
Overview
This checklist covers all testing requirements for the Postman Integration, including unit tests, integration tests, contract tests, performance tests, and compliance tests. Target throughput: 5000+ messages/second.
Total Items: 300+ checkboxes
Unit Tests
Message Envelope Tests (20 tests)
- [ ] Test envelope creation with all fields
- [ ] Test envelope creation with minimal fields
- [ ] Test envelope validation (valid case)
- [ ] Test envelope validation (missing required fields)
- [ ] Test envelope validation (invalid field types)
- [ ] Test envelope validation (invalid priority)
- [ ] Test envelope validation (invalid TTL)
- [ ] Test envelope serialization to JSON
- [ ] Test envelope deserialization from JSON
- [ ] Test envelope with binary payload
- [ ] Test envelope with JSON payload
- [ ] Test envelope with large payload (1MB)
- [ ] Test envelope with oversized payload (>1MB, should fail)
- [ ] Test envelope with empty payload
- [ ] Test envelope UUID generation uniqueness
- [ ] Test envelope timestamp format (ISO 8601)
- [ ] Test correlation_id propagation
- [ ] Test reply_to field handling
- [ ] Test content_type validation
- [ ] Test envelope immutability after creation
Queue Implementation Tests (30 tests)
- [ ] Test FIFO queue creation
- [ ] Test priority queue creation
- [ ] Test queue enqueue operation
- [ ] Test queue dequeue operation
- [ ] Test queue peek operation (no removal)
- [ ] Test queue size tracking
- [ ] Test queue empty check
- [ ] Test queue full check
- [ ] Test queue overflow behavior (bounded queue)
- [ ] Test queue underflow behavior (empty dequeue)
- [ ] Test queue with single element
- [ ] Test queue with 1000 elements
- [ ] Test queue with 10000 elements (max)
- [ ] Test queue message ordering (FIFO)
- [ ] Test queue priority ordering
- [ ] Test queue concurrent access (10 threads)
- [ ] Test queue concurrent access (100 threads)
- [ ] Test queue clear operation
- [ ] Test queue iteration
- [ ] Test queue persistence to disk
- [ ] Test queue restoration from disk
- [ ] Test queue corruption detection
- [ ] Test queue repair
- [ ] Test queue message TTL expiration
- [ ] Test queue message deduplication
- [ ] Test queue statistics (enqueue/dequeue counts)
- [ ] Test dead-letter queue integration
- [ ] Test queue batch operations
- [ ] Test queue transaction support
- [ ] Test queue memory limits
Topic Management Tests (25 tests)
- [ ] Test topic creation
- [ ] Test topic deletion
- [ ] Test topic listing
- [ ] Test topic name validation
- [ ] Test topic with no subscribers
- [ ] Test topic with single subscriber
- [ ] Test topic with 100 subscribers
- [ ] Test topic with 1000 subscribers
- [ ] Test exact topic match
- [ ] Test wildcard topic match (*)
- [ ] Test hierarchical topic match (topic.subtopic)
- [ ] Test regex topic pattern matching
- [ ] Test topic subscription
- [ ] Test topic unsubscription
- [ ] Test duplicate subscription prevention
- [ ] Test subscription idempotency
- [ ] Test topic persistence
- [ ] Test topic restoration
- [ ] Test topic metadata (created_at, subscriber_count)
- [ ] Test topic access control
- [ ] Test topic rate limiting
- [ ] Test topic message filtering
- [ ] Test cross-session topic isolation
- [ ] Test global topic (cross-session)
- [ ] Test topic statistics
Subscription Manager Tests (20 tests)
- [ ] Test subscription creation
- [ ] Test subscription deletion
- [ ] Test subscription listing by agent
- [ ] Test subscription listing by topic
- [ ] Test subscription pattern matching
- [ ] Test subscription with exact pattern
- [ ] Test subscription with wildcard pattern
- [ ] Test subscription with regex pattern
- [ ] Test subscription with hierarchical pattern
- [ ] Test subscription persistence
- [ ] Test subscription restoration
- [ ] Test subscription expiration (TTL)
- [ ] Test subscription renewal
- [ ] Test subscription metadata (created_at, last_match)
- [ ] Test subscription statistics
- [ ] Test subscription conflict detection
- [ ] Test subscription priority ordering
- [ ] Test subscription batching
- [ ] Test subscription index efficiency
- [ ] Test subscription concurrent access
Routing Engine Tests (30 tests)
- [ ] Test direct routing (agent-to-agent)
- [ ] Test topic-based routing
- [ ] Test queue-based routing
- [ ] Test broadcast routing
- [ ] Test multicast routing
- [ ] Test route resolution by agent ID
- [ ] Test route resolution by topic
- [ ] Test route resolution by filter
- [ ] Test route caching
- [ ] Test route cache invalidation
- [ ] Test route cache hit/miss
- [ ] Test routing rule evaluation
- [ ] Test routing rule priority
- [ ] Test routing rule conflicts
- [ ] Test content-based routing
- [ ] Test header-based routing
- [ ] Test sender-based routing
- [ ] Test conditional routing (if-then)
- [ ] Test load-balanced routing
- [ ] Test failover routing
- [ ] Test routing with unavailable recipient
- [ ] Test routing with multiple recipients
- [ ] Test routing performance (1000 routes)
- [ ] Test routing pattern compilation
- [ ] Test routing rule validation
- [ ] Test routing cycle detection
- [ ] Test routing loop prevention
- [ ] Test routing statistics
- [ ] Test routing audit logging
- [ ] Test cross-session routing prevention
Delivery Receipt Tests (15 tests)
- [ ] Test receipt generation on delivery
- [ ] Test receipt generation on failure
- [ ] Test receipt with delivery timestamp
- [ ] Test receipt with failure reason
- [ ] Test receipt storage
- [ ] Test receipt persistence
- [ ] Test receipt query by message ID
- [ ] Test receipt query by time range
- [ ] Test receipt query by recipient
- [ ] Test receipt expiration
- [ ] Test receipt statistics
- [ ] Test receipt acknowledgment
- [ ] Test receipt retry tracking
- [ ] Test receipt correlation with messages
- [ ] Test receipt export for audit
Error Handling Tests (20 tests)
- [ ] Test connection error handling
- [ ] Test timeout error handling
- [ ] Test validation error handling
- [ ] Test authorization error handling
- [ ] Test system error handling (OOM)
- [ ] Test system error handling (disk full)
- [ ] Test error message generation
- [ ] Test error code mapping
- [ ] Test error description clarity
- [ ] Test error stack trace inclusion (debug mode)
- [ ] Test error notification to sender
- [ ] Test error logging
- [ ] Test error log rotation
- [ ] Test transient error retry
- [ ] Test permanent error handling
- [ ] Test error alert generation
- [ ] Test graceful degradation on error
- [ ] Test circuit breaker activation
- [ ] Test circuit breaker recovery
- [ ] Test error statistics tracking
Integration Tests
Prolog Integration Tests (25 tests)
- [ ] Test Postman state sync to Prolog KB
- [ ] Test agent registration in Prolog
- [ ] Test queue creation in Prolog
- [ ] Test topic creation in Prolog
- [ ] Test subscription storage in Prolog
- [ ] Test routing rule storage in Prolog
- [ ] Test receipt storage in Prolog
- [ ] Test Prolog query for agent status
- [ ] Test Prolog query for subscriptions
- [ ] Test Prolog query for routing rules
- [ ] Test Prolog-based message filtering
- [ ] Test Prolog-based routing decisions
- [ ] Test Prolog KB unavailability handling
- [ ] Test Prolog KB recovery
- [ ] Test Prolog state reconciliation
- [ ] Test Prolog batch updates
- [ ] Test Prolog transaction support
- [ ] Test Prolog query caching
- [ ] Test Prolog cache invalidation
- [ ] Test Prolog performance (1000 facts)
- [ ] Test Prolog performance (10000 facts)
- [ ] Test Prolog concurrent access
- [ ] Test Prolog fact schema validation
- [ ] Test Prolog as single source of truth
- [ ] Test Prolog integration metrics
Agent Integration Tests (25 tests)
- [ ] Test agent registration with Postman
- [ ] Test agent deregistration
- [ ] Test agent message send
- [ ] Test agent message receive
- [ ] Test agent queue creation
- [ ] Test agent queue deletion
- [ ] Test agent topic subscription
- [ ] Test agent topic unsubscription
- [ ] Test agent status tracking (connected)
- [ ] Test agent status tracking (disconnected)
- [ ] Test agent status tracking (idle)
- [ ] Test agent status tracking (busy)
- [ ] Test agent capability declaration
- [ ] Test agent constraint enforcement
- [ ] Test agent rate limiting
- [ ] Test agent message size limits
- [ ] Test agent queue depth limits
- [ ] Test agent disconnection cleanup
- [ ] Test agent reconnection
- [ ] Test agent message queue preservation
- [ ] Test multi-agent message exchange
- [ ] Test agent broadcast
- [ ] Test agent multicast
- [ ] Test agent error notification
- [ ] Test agent metrics collection
Session Integration Tests (20 tests)
- [ ] Test session-based queue creation
- [ ] Test session-based topic creation
- [ ] Test session-based subscription
- [ ] Test session message isolation
- [ ] Test cross-session message prevention
- [ ] Test session queue naming (session-{id}-{name})
- [ ] Test session topic naming (session-{id}-{topic})
- [ ] Test session cleanup on end
- [ ] Test session queue purge on end
- [ ] Test session subscription removal on end
- [ ] Test session routing rule removal on end
- [ ] Test session statistics tracking
- [ ] Test session message history
- [ ] Test session state persistence
- [ ] Test session state restoration
- [ ] Test global topic access (if configured)
- [ ] Test shared queue access (if configured)
- [ ] Test session lifecycle hooks
- [ ] Test multiple concurrent sessions
- [ ] Test session isolation verification
Hook System Integration Tests (20 tests)
- [ ] Test pre_send hook execution
- [ ] Test post_send hook execution
- [ ] Test pre_deliver hook execution
- [ ] Test post_deliver hook execution
- [ ] Test on_delivery_failure hook execution
- [ ] Test on_message_expire hook execution
- [ ] Test pre_subscribe hook execution
- [ ] Test post_subscribe hook execution
- [ ] Test pre_enqueue hook execution
- [ ] Test post_enqueue hook execution
- [ ] Test hook execution order
- [ ] Test hook exception handling
- [ ] Test hook timeout (100ms limit)
- [ ] Test hook async execution
- [ ] Test hook state modification
- [ ] Test hook cancellation
- [ ] Test hook dynamic registration
- [ ] Test hook dynamic unregistration
- [ ] Test hook enable/disable
- [ ] Test hook metrics collection
Multi-Component Workflow Tests (30 tests)
- [ ] Test full message lifecycle (send -> route -> deliver -> ack)
- [ ] Test topic publish workflow
- [ ] Test queue workflow (enqueue -> dequeue)
- [ ] Test priority message workflow
- [ ] Test broadcast workflow
- [ ] Test request-response workflow (correlation_id)
- [ ] Test message retry workflow
- [ ] Test message expiration workflow
- [ ] Test dead-letter queue workflow
- [ ] Test circuit breaker workflow
- [ ] Test agent reconnection workflow
- [ ] Test session cleanup workflow
- [ ] Test message persistence workflow
- [ ] Test message replay workflow
- [ ] Test routing rule update workflow
- [ ] Test subscription update workflow
- [ ] Test configuration reload workflow
- [ ] Test graceful shutdown workflow
- [ ] Test system recovery workflow
- [ ] Test load shedding workflow
- [ ] Test backpressure workflow
- [ ] Test metric collection workflow
- [ ] Test log aggregation workflow
- [ ] Test alert generation workflow
- [ ] Test health check workflow
- [ ] Test admin operation workflow
- [ ] Test audit trail workflow
- [ ] Test compliance check workflow
- [ ] Test version negotiation workflow
- [ ] Test migration workflow
Contract Tests
Message Envelope Contract (10 tests)
- [ ] Verify envelope schema matches specification
- [ ] Verify all required fields present
- [ ] Verify field types match specification
- [ ] Verify field constraints enforced
- [ ] Verify JSON serialization format
- [ ] Verify timestamp format (ISO 8601)
- [ ] Verify UUID format (RFC 4122)
- [ ] Verify priority range (0-4)
- [ ] Verify TTL range (positive integer)
- [ ] Verify envelope version compatibility
Queue API Contract (10 tests)
- [ ] Verify enqueue API signature
- [ ] Verify dequeue API signature
- [ ] Verify peek API signature
- [ ] Verify queue creation API signature
- [ ] Verify queue deletion API signature
- [ ] Verify queue listing API signature
- [ ] Verify queue statistics API signature
- [ ] Verify API return types
- [ ] Verify API error responses
- [ ] Verify API backward compatibility
Topic API Contract (10 tests)
- [ ] Verify subscribe API signature
- [ ] Verify unsubscribe API signature
- [ ] Verify topic creation API signature
- [ ] Verify topic deletion API signature
- [ ] Verify topic listing API signature
- [ ] Verify topic query API signature
- [ ] Verify publish API signature
- [ ] Verify API return types
- [ ] Verify API error responses
- [ ] Verify API backward compatibility
Receipt API Contract (5 tests)
- [ ] Verify receipt schema matches specification
- [ ] Verify receipt fields present
- [ ] Verify receipt query API signature
- [ ] Verify receipt storage format
- [ ] Verify receipt backward compatibility
Metrics API Contract (10 tests)
- [ ] Verify Prometheus format compliance
- [ ] Verify metric naming convention
- [ ] Verify metric labels
- [ ] Verify metric types (counter/gauge/histogram)
- [ ] Verify histogram buckets
- [ ] Verify JSON metrics format
- [ ] Verify real-time streaming format
- [ ] Verify metric aggregation
- [ ] Verify metric retention policy
- [ ] Verify metrics endpoint availability
Prolog Fact Contract (10 tests)
- [ ] Verify postman_agent fact schema
- [ ] Verify postman_queue fact schema
- [ ] Verify postman_topic fact schema
- [ ] Verify postman_subscription fact schema
- [ ] Verify postman_route fact schema
- [ ] Verify postman_receipt fact schema
- [ ] Verify postman_stats fact schema
- [ ] Verify fact naming convention
- [ ] Verify fact query patterns
- [ ] Verify fact update patterns
Performance Tests
Throughput Tests (Target: 5000 msg/sec)
- [ ] Test 100 messages/second sustained (1 minute)
- [ ] Test 500 messages/second sustained (1 minute)
- [ ] Test 1000 messages/second sustained (1 minute)
- [ ] Test 2000 messages/second sustained (1 minute)
- [ ] Test 3000 messages/second sustained (1 minute)
- [ ] Test 4000 messages/second sustained (1 minute)
- [ ] Test 5000 messages/second sustained (1 minute)
- [ ] Test 6000 messages/second sustained (1 minute)
- [ ] Test 7000 messages/second sustained (1 minute)
- [ ] Test 8000 messages/second sustained (1 minute)
- [ ] Test 5000 messages/second sustained (5 minutes)
- [ ] Test 5000 messages/second sustained (30 minutes)
- [ ] Test 5000 messages/second sustained (1 hour)
- [ ] Verify no message loss at target throughput
- [ ] Verify message ordering at target throughput
Latency Tests (Target: <100ms p95)
- [ ] Measure send latency (p50, p95, p99)
- [ ] Measure delivery latency (p50, p95, p99)
- [ ] Measure end-to-end latency (p50, p95, p99)
- [ ] Measure queue latency (p50, p95, p99)
- [ ] Measure topic publish latency (p50, p95, p99)
- [ ] Measure subscription match latency (p50, p95, p99)
- [ ] Measure routing decision latency (p50, p95, p99)
- [ ] Measure acknowledgment latency (p50, p95, p99)
- [ ] Verify p95 latency < 100ms
- [ ] Verify p99 latency < 200ms
- [ ] Test latency under load (5000 msg/sec)
- [ ] Test latency with large payloads (1MB)
- [ ] Test latency with many subscribers (100+)
- [ ] Test latency with complex routing (1000+ rules)
- [ ] Test latency with Prolog integration
Scalability Tests
- [ ] Test with 10 concurrent agents
- [ ] Test with 100 concurrent agents
- [ ] Test with 1000 concurrent agents
- [ ] Test with 10 active queues
- [ ] Test with 100 active queues
- [ ] Test with 1000 active queues
- [ ] Test with 10 active topics
- [ ] Test with 100 active topics
- [ ] Test with 1000 active topics
- [ ] Test with 10 subscriptions per topic
- [ ] Test with 100 subscriptions per topic
- [ ] Test with 1000 subscriptions per topic
- [ ] Test with 100 routing rules
- [ ] Test with 1000 routing rules
- [ ] Test with 10000 routing rules
Resource Usage Tests
- [ ] Measure memory usage at idle
- [ ] Measure memory usage at 1000 msg/sec
- [ ] Measure memory usage at 5000 msg/sec
- [ ] Verify memory usage < 1GB at target throughput
- [ ] Measure CPU usage at idle
- [ ] Measure CPU usage at 1000 msg/sec
- [ ] Measure CPU usage at 5000 msg/sec
- [ ] Verify CPU usage < 50% at target throughput
- [ ] Measure disk I/O at 5000 msg/sec
- [ ] Measure network I/O at 5000 msg/sec
- [ ] Measure thread pool utilization
- [ ] Test memory leak detection (24-hour run)
- [ ] Test file descriptor leak detection
- [ ] Test connection pool exhaustion
- [ ] Test queue memory limits
Stress Tests
- [ ] Test with 10x target throughput (50000 msg/sec)
- [ ] Test with 1000 concurrent connections
- [ ] Test with 10000 concurrent connections
- [ ] Test queue overflow behavior
- [ ] Test memory exhaustion behavior
- [ ] Test disk full behavior
- [ ] Test CPU saturation behavior
- [ ] Test network saturation behavior
- [ ] Test simultaneous component failure
- [ ] Test cascading failure scenarios
- [ ] Test recovery from stress conditions
- [ ] Test graceful degradation under stress
Endurance Tests
- [ ] Run at target throughput for 1 hour
- [ ] Run at target throughput for 6 hours
- [ ] Run at target throughput for 24 hours
- [ ] Run at target throughput for 7 days
- [ ] Monitor for memory leaks
- [ ] Monitor for performance degradation
- [ ] Monitor for connection leaks
- [ ] Monitor for disk space growth
- [ ] Verify system stability over time
- [ ] Verify consistent latency over time
Spike Tests
- [ ] Test sudden load increase (0 -> 5000 msg/sec)
- [ ] Test sudden load increase (1000 -> 10000 msg/sec)
- [ ] Test load drop (5000 -> 0 msg/sec)
- [ ] Test load spike recovery
- [ ] Test repeated load spikes (every 5 minutes)
- [ ] Test connection burst (1000 simultaneous connects)
- [ ] Test subscription burst (10000 simultaneous subscribes)
- [ ] Test queue creation burst (1000 simultaneous creates)
Compliance Tests
Constitutional Compliance (11 tests)
- [ ] Test agent autonomy compliance (Article I)
- [ ] Test Prolog as source of truth (Article II)
- [ ] Test session isolation (Article III)
- [ ] Test hook extensibility (Article IV)
- [ ] Test core/extension separation (Article V)
- [ ] Test Python 3.11+ compliance (Article VI)
- [ ] Test no-mocking policy (Article VII)
- [ ] Test metrics/observability (Article VIII)
- [ ] Test graceful degradation (Article IX)
- [ ] Test version compatibility (Article X)
- [ ] Test documentation completeness (Article XI)
Security Compliance
- [ ] Test session isolation enforcement
- [ ] Test cross-session access prevention
- [ ] Test authentication requirement
- [ ] Test authorization checks
- [ ] Test input validation
- [ ] Test injection attack prevention
- [ ] Test DoS attack prevention
- [ ] Test rate limiting
- [ ] Test resource quotas
- [ ] Test audit logging
Data Integrity
- [ ] Test message ordering preservation
- [ ] Test message deduplication
- [ ] Test message corruption detection
- [ ] Test message integrity verification
- [ ] Test queue consistency
- [ ] Test topic consistency
- [ ] Test subscription consistency
- [ ] Test Prolog KB consistency
- [ ] Test transaction atomicity
- [ ] Test crash recovery
Failure Scenario Tests
Network Failures
- [ ] Test agent disconnection during send
- [ ] Test agent disconnection during delivery
- [ ] Test network partition
- [ ] Test network timeout
- [ ] Test connection refused
- [ ] Test DNS failure
- [ ] Test packet loss simulation
- [ ] Test bandwidth limitation
System Failures
- [ ] Test out-of-memory condition
- [ ] Test disk full condition
- [ ] Test disk read error
- [ ] Test disk write error
- [ ] Test CPU exhaustion
- [ ] Test thread pool exhaustion
- [ ] Test file descriptor exhaustion
- [ ] Test process crash
Component Failures
- [ ] Test Prolog KB unavailability
- [ ] Test Prolog KB corruption
- [ ] Test queue corruption
- [ ] Test topic registry corruption
- [ ] Test subscription index corruption
- [ ] Test routing table corruption
- [ ] Test receipt store corruption
- [ ] Test log file corruption
Recovery Tests
- [ ] Test recovery from agent disconnection
- [ ] Test recovery from Prolog KB failure
- [ ] Test recovery from disk full
- [ ] Test recovery from memory exhaustion
- [ ] Test recovery from process crash
- [ ] Test recovery from network partition
- [ ] Test state reconstruction
- [ ] Test message queue recovery
- [ ] Test subscription recovery
- [ ] Test routing rule recovery
Completion Summary
Total Checkboxes: 500+ Completed: 0 In Progress: 0 Not Started: 500+
Progress: 0%
Test Execution Guidelines
Test Environment
- Use real components (no mocking per Article VII)
- Use isolated test environments
- Clean state between tests
- Use temporary directories for persistence
- Use separate Prolog KB for tests
Performance Test Requirements
- Run on production-like hardware
- Disable debug logging during performance tests
- Use realistic message payloads
- Monitor resource usage during tests
- Generate performance reports
Test Reporting
- Report pass/fail for each test
- Report performance metrics
- Report resource usage
- Report anomalies and failures
- Generate test coverage report
Continuous Testing
- Run unit tests on every commit
- Run integration tests on every PR
- Run performance tests weekly
- Run endurance tests monthly
- Run compliance tests on release
Notes
- All tests must use real implementations (no mocking)
- Performance tests must meet 5000 msg/sec target
- Latency tests must meet <100ms p95 target
- All constitutional compliance tests must pass
- Failure scenarios must be tested systematically
Other tracked files
specs/002-postman/compliance-audit.mdspecs/002-postman/review-2026-03-25.md
Full spec text
Verbatim from specs/002-postman/spec.md; headings are pushed down two levels.
SPEC-002: Postman Message Broker (U:PS)
Metadata
- Spec ID: 002-postman
- Title: Postman Message Broker
- Alias: U:PS
- Author: Mohamed
- Status: Complete
- Created: 2026-01-13
- Last Updated: 2026-01-15
- Priority: P0
- Components: Postman module, Message Queue, Topic Manager
Overview
What
The Postman (U:PS) is the message routing engine for Orchestrata. It handles message delivery, topic-based subscriptions, queue management, priority processing, message filtering, persistence, and delivery guarantees.
Why
- Reliable message delivery between agents
- Decoupled communication via topics and queues
- Delivery guarantees (at-most-once, at-least-once, exactly-once)
- Message persistence for recovery and replay
- Performance: 5000+ messages/second
How
- Redis-backed queues and topics
- Multiple delivery protocols (CXP, gRPC, WebSocket, HTTP)
- Priority-based message processing
- Acknowledgment-based delivery guarantees
User Scenarios & Stories
US-001: Agent Message Delivery (Critical)
As an agent I want to send messages through the Postman service So that they are reliably delivered to recipients
Acceptance Criteria:
- Send messages via Postman API
- Messages routed to correct recipient agents
- Delivery acknowledged with receipts
- Failed deliveries trigger retry (up to 3 attempts)
- Delivery status tracked and logged
- Support all CXP message types (INFORM, REQUEST, QUERY, RESPONSE, ERROR, NOTIFY, SUBSCRIBE, UNSUBSCRIBE)
- Delivery < 100ms (p95)
- Support 5000+ messages/second throughput
US-002: Topic-Based Message Subscription (Critical)
As an agent I want to subscribe to topics So that I receive relevant messages without direct addressing
Acceptance Criteria:
- Subscribe/unsubscribe from topics
- List subscribed topics
- Support wildcard patterns (e.g., "agent.*")
- Support hierarchical topic structures
- Messages delivered to all topic subscribers
- Persist subscriptions across restarts
- Auto-unsubscribe on agent termination
- Subscribe/unsubscribe < 50ms
US-003: Message Queue Management (High)
As an agent I want to use message queues So that I can buffer and process messages asynchronously
Acceptance Criteria:
- Create named message queues
- FIFO and priority queue support
- Message deduplication
- Dead-letter queue for failed messages
- Message TTL (time-to-live)
- Single and multiple consumer modes
- Peek messages without consuming
- Batch message retrieval
- Max queue size: 10,000 messages
- Persist queue state to disk
US-004: Message Priority Handling (High)
As a sender I want to assign priorities to messages So that urgent messages are processed first
Acceptance Criteria:
- Priority levels: CRITICAL (0), HIGH (1), NORMAL (2), LOW (3), BACKGROUND (4)
- Process messages in priority order
- Prevent starvation of low-priority messages
- Age-based priority boost
- Priority inversion detection
- Track processing time per priority
US-005: Message Filtering and Routing (High)
As a system administrator I want to filter and route messages So that messages reach the right destinations
Acceptance Criteria:
- Content-based filtering
- Header-based filtering
- Pattern-based filtering (regex)
- Composite filters (AND/OR logic)
- Static and dynamic routing rules
- Conditional routing (if-then rules)
- Load-balanced and failover routing
- Rule evaluation < 10ms per message
- Support 1000+ active rules
US-006: Message Persistence and Replay (High)
As a developer I want to persist and replay messages So that I can recover from failures and debug issues
Acceptance Criteria:
- Persist all messages to disk
- Store metadata, payload, delivery receipts
- Configurable retention period (default 7 days)
- Replay messages by time range, sender, topic
- Replay at original or accelerated speed
- Dry-run replay (no delivery)
- Recover from storage corruption
US-007: Delivery Guarantees and Acknowledgments (Critical)
As a sender I want to configure delivery guarantees So that messages are delivered reliably
Acceptance Criteria:
- At-most-once delivery (fire and forget)
- At-least-once delivery (with retries)
- Exactly-once delivery (with deduplication)
- Synchronous and asynchronous acknowledgments
- Negative ACK (NACK) for failures
- Batch and selective acknowledgments
- Configurable ACK timeout
- Exponential backoff for retries
- Dead-letter queue for undeliverable messages
US-008: Monitoring and Metrics (High)
As a system operator I want to monitor the Postman service So that I can ensure system health
Acceptance Criteria:
- Messages sent/received per second
- Messages queued and in-flight
- Average message latency (p50, p95, p99)
- Active agents count
- Queue depth per queue
- Topic subscribers count
- CPU, memory, disk, network metrics
- Prometheus format export
- Real-time metrics streaming
US-009: Configuration Management (Medium)
As a system administrator I want to configure the Postman service So that I can tune for my environment
Acceptance Criteria:
- Load from YAML, environment variables, CLI arguments
- Configure service port, max message size, queue size
- Configure retry policy, thread pool size
- Topic and queue-specific settings
- Security configuration (auth, encryption)
- Hot-reload without restart
US-010: Performance and Scalability (Critical)
As a system architect I want high performance and scalability So that the system handles load
Acceptance Criteria:
- Handle 5000+ messages/second
- Handle 10,000+ concurrent connections
- Handle 1000+ active queues
- Handle 100+ topics with subscribers
- Message delivery < 100ms (p95)
- Queue operations < 50ms (p95)
- Memory usage < 1GB
- Efficient CPU utilization (<50% at peak)
Functional Requirements
See checklists/requirements.md for 200+ detailed requirements.
Key Performance Targets
| Operation | Target |
|---|---|
| Message Delivery | < 100ms (p95) |
| Queue Operations | < 50ms (p95) |
| Topic Subscription | < 50ms (p95) |
| ACK Processing | < 10ms (p95) |
| Throughput | 5000+ msg/sec |
Technical Design
Architecture
┌─────────────────────────────────────────────────────────┐
│ U:PS - Postman │
├─────────────────────────────────────────────────────────┤
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ Message │ │ Topic │ │ Queue │ │
│ │ Router │ │ Manager │ │ Manager │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │ │
│ ┌──────┴──────┐ ┌──────┴──────┐ ┌──────┴──────┐ │
│ │ Delivery │ │ Subscription│ │ Persistence │ │
│ │ Engine │ │ Manager │ │ Manager │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
├─────────────────────────────────────────────────────────┤
│ Redis (Queues/Topics) │ Disk (Persistence) │
└─────────────────────────────────────────────────────────┘
Message Envelope
class MessageEnvelope:
message_id: UUID
sender: str # Agent ID
recipient: str # Agent ID or topic
timestamp: datetime # ISO 8601
priority: int # 0-4
ttl: int # seconds
correlation_id: Optional[str] # For request-response
reply_to: Optional[str] # For responses
content_type: str # MIME type
payload: Union[dict, bytes]
Delivery Semantics
class DeliveryMode(Enum):
AT_MOST_ONCE = "fire_and_forget"
AT_LEAST_ONCE = "with_retries"
EXACTLY_ONCE = "with_dedup"
Integration Points
| Component | Integration |
|---|---|
| orchestrata-kernel | Message routing decisions |
| orchestrata-cxp | CXP packet handling |
| orchestrata-governance | Delivery authorization |
| orchestrata-session | Session-based isolation |
Success Criteria
- Performance: 5000+ msg/sec, <100ms delivery latency
- Reliability: 99.99% message delivery rate
- Scale: 10,000+ concurrent connections
- Durability: Zero message loss with persistence enabled
- Flexibility: Support all three delivery guarantees
Dependencies
- orchestrata-core (enums, interfaces)
- orchestrata-cxp (packet format)
- orchestrata-kernel (routing decisions)
- Redis (queues, topics, state)
Revision History
| Date | Version | Author | Changes |
|---|---|---|---|
| 2026-03-07 | 1.0 | Mohamed | Initial specification |