Show & Tell: Phase 11.1 — SafetyFilter (constitutional ruleset, BLOCK/CRITICAL verdicts, autonomy loop pause)

[web3guru888]

Phase 11.1 — SafetyFilter architecture: constitutional ruleset gating for autonomous goal execution

Issue: #337
Phase: 11.1 (Safety & Alignment)
Status: In spec

---

Why SafetyFilter?

Phase 10 closes the self-management loop. But a fully autonomous agent that accepts and executes any goal it receives is unsafe by design. SafetyFilter is the alignment layer — every Goal and every SubTask passes through a constitutional ruleset before it can enter the Phase 10 pipeline.

---

Component Map

External Input / API
      │
      ▼
SafeGoalRegistry.register(goal)
      │  ① check_goal(goal) → SafetyVerdict
      ▼
InMemorySafetyFilter ◄──── ConstitutionalRuleset (SR-001 … SR-007)
      │                         + custom rules (register_rule)
      │  BLOCK / CRITICAL → raise SafetyBlockedError
      │  WARN / INFO → proceed, log violation
      ▼
GoalRegistry.register(goal)  ← only if verdict.allowed
      │
      ▼
GoalDecomposer.decompose(goal) → TaskGraph
      │
      ▼
SafeGoalDecomposer ──── ② check_task_graph(graph)
      │  SR-005: subtask count, SR-003/004: per-subtask caps
      │  BLOCK → raise RuntimeError (replan or abandon)
      ▼
PlanExecutor → ExecutionMonitor → ReplanningEngine

---

The 7 Constitutional Rules

Rule	Target	Severity	Logic
SR-001	Goal	BLOCK	len(description) > max_goal_description_len (2048)
SR-002	Goal	BLOCK	priority == CRITICAL without allow_critical=True
SR-003	SubTask	BLOCK	required_caps contains a blocked pattern (network:external, exec:arbitrary)
SR-004	SubTask	WARN	required_caps prefix not in allowed_capability_prefixes
SR-005	TaskGraph	BLOCK	Total subtask count > max_subtasks_per_graph (128)
SR-006	Goal	WARN	Description matches https?:// regex (potential exfiltration signal)
SR-007	Goal	CRITICAL	goal_type in hardcoded denylist (disable_safety_filter, self_modify_runtime, exfiltrate_data)

---

check_goal() Flow

# Simplified
async def check_goal(self, goal: Goal) -> SafetyVerdict:
    viols = []
    for rule, predicate in self._rules:
        try:
            if predicate(goal):
                v = SafetyViolation(rule.rule_id, rule.severity, goal.goal_id, "goal", rule.description)
                viols.append(v)
                if rule.severity == ViolationSeverity.CRITICAL and self._config.pause_on_critical:
                    self._paused = True          # freeze autonomy loop
                    _safety_paused.set(1)        # Prometheus alert
        except Exception:
            pass  # predicate errors must never block execution
    # Record + metrics
    async with self._lock:
        self._violations.extend(viols)
    allowed = not any(v.severity in (BLOCK, CRITICAL) for v in viols)
    return SafetyVerdict(goal.goal_id, allowed, tuple(viols))

Key point: predicates are wrapped in try/except. A buggy custom rule never causes a safety check failure — it silently passes. This is intentional: fail open for rule errors, fail closed for policy violations.

---

ViolationSeverity — escalation model

INFO ──► log only (no impact on verdict.allowed)
WARN ──► log + flag (no impact on verdict.allowed)
BLOCK ──► allowed = False (goal/subtask rejected)
CRITICAL ──► allowed = False + _paused = True (autonomy loop frozen)

A CRITICAL violation immediately sets _paused = True. CognitiveCycle should check safety_filter._paused at each tick and skip goal processing until an operator clears the pause.

---

SafetyVerdict data model

@dataclass(frozen=True)
class SafetyVerdict:
    subject_id: str
    allowed: bool                          # False → reject
    violations: tuple[SafetyViolation, ...]
    evaluated_at: float                    # monotonic timestamp

Multiple violations can coexist. A goal can have SR-006 (WARN) and SR-001 (BLOCK) simultaneously — the verdict is blocked because of SR-001, but SR-006 is also recorded.

---

5 Prometheus Metrics

Metric	Type	Key Labels
asi_safety_checks_total	Counter	subject_type (goal/subtask/graph), result (allowed/blocked)
asi_safety_violations_total	Counter	rule_id, severity
asi_safety_paused	Gauge	—
asi_safety_rules_registered	Gauge	—
asi_safety_check_duration_seconds	Histogram	subject_type

PromQL — critical violation alert:

increase(asi_safety_violations_total{severity="critical"}[1m]) > 0

---

Open Questions

1. Predicate persistence: Rule predicates are lambdas — they cannot be serialized to FederatedBlackboard. Should safety rules be stored as named rule IDs with a registry lookup, rather than raw callables?

2. Async predicates: Some rules (e.g. "is this peer in the federation allowlist?") require async lookups. Should the predicate signature be async (goal) -> bool?

3. Pause recovery: When _paused = True fires, how should it be cleared? Options: (a) operator API call safety_filter.resume(), (b) automatic after N seconds, (c) require all CRITICAL violations to be acknowledged.

---

Issue #337 | Phase 11 planning → #336 | Phase 10.5 ReplanningEngine → #333