Phase 17.2 Show & Tell — EventSequencer: streaming causal event processor architecture

[web3guru888]

EventSequencer Architecture — Phase 17.2 Show & Tell 🕐

Issue: #437 | Phase: 17 — Temporal Reasoning & Predictive Cognition

---

Full Data Flow

┌─────────────────────────────────────────────────────────────────┐
│                    CognitiveCycle._run_step()                    │
│                                                                  │
│  module_a.run() ──┐                                             │
│  module_b.run() ──┼──► emit CognitiveEvent(ts=T, parent=P) ──► │
│  module_c.run() ──┘                                             │
└──────────────────────────────────┬──────────────────────────────┘
                                   │  raw, unordered events
                                   ▼
┌──────────────────────────────────────────────────────────────────┐
│                    AsyncEventSequencer                            │
│                                                                  │
│   ingest(event)                                                  │
│     ├── causal_check: parent_id ∈ seen_ids? ──► violation cnt   │
│     ├── overflow check: len(heap) >= max_buffer ──► LRU evict   │
│     ├── heapq.heappush(heap, event)  [O(log n)]                 │
│     └── _accumulate_window(event)   [bucket assignment]          │
│                                                                  │
│   drain()  [called from CognitiveCycle]                          │
│     └── while heap: yield heapq.heappop(heap)  [timestamp order] │
│                                                                  │
│   flush_windows()  [called periodically]                         │
│     └── return completed WindowedAggregates                      │
└──────────┬──────────────────────────┬───────────────────────────┘
           │ ordered CognitiveEvents  │ WindowedAggregates
           ▼                          ▼
┌─────────────────────┐   ┌──────────────────────────┐
│   TemporalGraph     │   │     PredictiveEngine      │
│  .add_node(e.id,    │   │   (Phase 17.3)            │
│   e.timestamp_ns)   │   │  consumes window batches  │
└─────────────────────┘   └──────────────────────────┘

---

OrderPolicy Behaviour Table

Policy	Out-of-order event	Causal violation	Use case
STRICT	❌ Dropped immediately	❌ Dropped, counter++	Safety-critical decision loops
RELAXED	✅ Reordered in heapq	⚠️ Accepted, counter++	Standard cognitive pipeline
BEST_EFFORT	✅ Accepted as-arrived	✅ Ignored	Telemetry / low-priority events

---

Tumbling vs Sliding Window Comparison

Timeline: ──────────────────────────────────────────────────────►

TUMBLING (window_size_ms=500):
  │←── 500ms ──►│←── 500ms ──►│←── 500ms ──►│
  [  Window 1   ][  Window 2   ][  Window 3   ]
  Events: e1 e2   Events: e3     Events: e4 e5
  Clean boundaries → predictable PredictiveEngine input ✅

SLIDING (step_ms=250, size_ms=500):
  │←───── 500ms ─────►│
        │←───── 500ms ─────►│
              │←───── 500ms ─────►│
  e1 e2 appear in Windows 1 AND 2 → duplicate processing ❌

Decision: Tumbling windows for deterministic batch delivery to PredictiveEngine.

---

Causal Ordering Validation Algorithm

FUNCTION validate_causal(event, seen_ids, policy):
  IF event.causal_parent_id IS NULL:
    RETURN ACCEPTED              # root event, no parent needed

  IF event.causal_parent_id IN seen_ids:
    RETURN ACCEPTED              # parent arrived before child ✅

  # Parent not yet seen
  causal_violations_total.inc(source_module=event.source_module)

  IF policy == STRICT:
    events_dropped_total.inc(reason="causal")
    RETURN DROPPED               # reject orphan in strict mode

  IF policy == RELAXED:
    RETURN ACCEPTED_WITH_WARNING # accept but flag for review

  IF policy == BEST_EFFORT:
    RETURN ACCEPTED              # causal order ignored

TTL pruning for seen_ids: In production, seen_ids should be an OrderedDict pruned to max_buffer * 2 entries to prevent unbounded memory growth.

---

Prometheus Metrics

Metric	Type	Key Labels
asi_sequencer_events_ingested_total	Counter	source_module, event_type
asi_sequencer_events_dropped_total	Counter	reason
asi_sequencer_windows_flushed_total	Counter	—
asi_sequencer_causal_violations_total	Counter	source_module
asi_sequencer_buffer_utilization_ratio	Gauge	—

PromQL snippets:

# Event throughput rate
rate(asi_sequencer_events_ingested_total[1m])

# Drop rate by reason
rate(asi_sequencer_events_dropped_total[5m]) by (reason)

# Causal violation rate (alert if > 5/min)
rate(asi_sequencer_causal_violations_total[1m]) > 0.083

# Buffer saturation (alert if > 80%)
asi_sequencer_buffer_utilization_ratio > 0.8

# Window flush rate
rate(asi_sequencer_windows_flushed_total[1m])

---

Open Questions

1. Distributed event ordering: When Phase 9 federation nodes emit events to a shared sequencer, how do we reconcile monotonic clocks across machines? (NTP jitter can exceed window_size_ms for small windows)

2. Window size tuning: 500ms default is chosen for human-scale cognition cycles. For sub-100ms reflexive responses (Phase 11 SafetyFilter), should window_size_ms be configurable per event_type?

3. Causal graph reconstruction: Can a sequence of CognitiveEvent.causal_parent_id chains reconstruct the full TemporalGraph without explicit Allen-relation labelling? (Useful for replay/debugging)

---

Part of Phase 17 — Temporal Reasoning & Predictive Cognition. Previous: #435 (TemporalGraph Show & Tell)