Show & Tell: AlignmentDashboard — Real-Time Operator Safety Console (Phase 11.5)

[web3guru888]

AlignmentDashboard — Architecture Overview

Phase 11 closes with the AlignmentDashboard: an SSE-backed operator console that aggregates live safety signals from all four preceding Phase 11 components into a single observable feed.

Spec: Issue #349

---

Component Map

┌──────────────────────────────────────────────────────────────────┐
│                       CognitiveCycle                             │
│  SafetyFilter ─── SAFETY_VIOLATION ──────────────────────┐       │
│  AlignmentMonitor ─ ALIGNMENT_SCORE (heartbeat 5 s) ─────┤       │
│  ValueLearner ──── REWARD_WEIGHTS   (heartbeat 15 s) ────┤       │
│  InterpretabilityProbe ── EXPLANATION (on block) ─────────┤      │
│                                                           ▼       │
│                    ┌─────────────────────┐                        │
│                    │  AlignmentDashboard  │                       │
│                    │  _broadcast() fanout │                       │
│                    └──────────┬──────────┘                        │
│                               │ SSE /alignment/stream             │
│                    ┌──────────▼──────────┐                        │
│                    │   Operator Browser   │                       │
│                    │  EventSource(url)    │                       │
│                    │  POST /override      │                       │
│                    └─────────────────────┘                        │
└──────────────────────────────────────────────────────────────────┘

---

Event Types

DashboardEventType	Source	Cadence
ALIGNMENT_SCORE	AlignmentMonitor snapshot	Every 5 s (heartbeat)
SAFETY_VIOLATION	SafetyFilter block	On every blocked goal
EXPLANATION	InterpretabilityProbe	On every blocked goal (paired with SAFETY_VIOLATION)
REWARD_WEIGHTS	ValueLearner snapshot	Every 15 s
OPERATOR_OVERRIDE	POST /alignment/override	On operator action

---

SSE Payload Examples

ALIGNMENT_SCORE

{
  "scores": {
    "GOAL_ALIGNMENT": 0.91,
    "HARM_AVOIDANCE": 0.88,
    "VALUE_CONSISTENCY": 0.84,
    "TRANSPARENCY": 0.95,
    "HUMAN_OVERSIGHT": 0.79
  },
  "overall": 0.874,
  "consecutive_fails": 0
}

SAFETY_VIOLATION

{
  "goal_id": "g-42",
  "verdict": "BLOCKED",
  "filters_triggered": ["HARM_AVOIDANCE", "VALUE_CONSISTENCY"],
  "score": 0.31,
  "counterfactual": "Reframe goal to exclude resource depletion constraint"
}

EXPLANATION (paired with SAFETY_VIOLATION)

{
  "goal_id": "g-42",
  "target": "SAFETY_FILTER",
  "method": "PERMUTATION",
  "attributions": [
    {"feature": "harm_score",           "weight": 0.38},
    {"feature": "value_consistency",    "weight": 0.27},
    {"feature": "scope_creep_flag",     "weight": 0.19}
  ]
}

OPERATOR_OVERRIDE

{
  "goal_id": "g-42",
  "action": "approve",
  "reason": "Manual safety review passed — low-risk context confirmed",
  "operator": "alice@ops"
}

---

Fanout Design

Each browser session gets its own asyncio.Queue[DashboardEvent] + deque(maxlen=1000) ring buffer:

# _broadcast — O(sessions) fanout
async def _broadcast(self, event: DashboardEvent) -> None:
    for sid, q in self._queues.items():
        self._sessions[sid].append(event)  # ring buffer (auto-evicts oldest)
        try:
            q.put_nowait(event)
        except asyncio.QueueFull:
            pass  # slow consumer — drop silently; metric: queue_depth

Why not a pub-sub library? For operator tools at 1–10 concurrent sessions the overhead of asyncio Queues is negligible. A Redis pub-sub back-end can replace _queues for multi-process deployments without changing the Protocol.

---

Prometheus Metrics

Metric	Type	Labels
asi_alignment_dashboard_events_total	Counter	event_type
asi_alignment_dashboard_sessions_active	Gauge	—
asi_alignment_dashboard_queue_depth	Gauge	session_id
asi_alignment_dashboard_overrides_total	Counter	action
asi_alignment_dashboard_heartbeat_latency_seconds	Histogram	—

PromQL — violations per minute

rate(asi_alignment_dashboard_events_total{event_type="SAFETY_VIOLATION"}[1m]) * 60

PromQL — active operator sessions

asi_alignment_dashboard_sessions_active

---

Open Questions

1. Persistence: Should DashboardEvent be written to a time-series store (e.g., InfluxDB) for post-hoc audit beyond the in-memory ring buffer?
2. Auth: JWT bearer token gating is configurable — what scopes should approve vs view operators have?
3. Multi-process: How should the dashboard scale beyond a single Python process — Redis pub-sub vs message broker?

Phase 11 is now fully spec'd — implementation PRs welcome!