Show & Tell: HotSwapper — zero-downtime live module swapping architecture

[web3guru888]

What is HotSwapper?

Phase 15.2 introduces HotSwapper — the component that makes ASI:BUILD truly self-modifying at runtime. Where ModuleRegistry (15.1) tracks what versions exist, HotSwapper decides when and how to apply them with full safety guarantees.

---

The 5-Phase Swap Lifecycle

Every call to HotSwapper.swap() passes through these stages:

PENDING -> VALIDATING -> SWAPPING -> VERIFYING -> COMMITTED
                                               \-> ROLLED_BACK (verify failure)
                                   (any exception) -> FAILED

Phase	What happens	Failure action
VALIDATING	registry.get_version → must be STAGED	raise ValueError
SWAPPING	cycle.set_module(name, new_impl)	raise → FAILED
VERIFYING	impl.health_check() duck-type probe	rollback if rollback_on_verify_fail=True
COMMITTED	registry.set_status → ACTIVE; old → ARCHIVED	—
ROLLED_BACK	cycle.set_module restored; registry → REVERTED	—

---

Concurrency Model

Two-layer locking ensures both safety and throughput:

# Layer 1: per-module Lock — serialises swaps of THE SAME module
self._locks: dict[str, asyncio.Lock] = defaultdict(asyncio.Lock)

# Layer 2: global Semaphore — caps total concurrent swaps (memory pressure)
self._sem = asyncio.Semaphore(self._cfg.max_concurrent_modules)  # default: 8

This means:

• Module A and Module B can swap in parallel ✅
• Two concurrent swaps of Module A run sequentially ✅
• Never more than 8 swaps active simultaneously ✅

---

Data Model (all frozen dataclasses)

@dataclass(frozen=True)
class SwapRequest:
    module_name:    str
    target_version: int
    initiator:      str   = "CognitiveCycle"
    timeout_s:      float = 10.0

@dataclass(frozen=True)
class SwapResult:
    module_name:  str
    from_version: int
    to_version:   int
    phase:        SwapPhase   # terminal phase
    duration_s:   float
    error:        str | None = None

---

CognitiveCycle Integration

async def _synthesis_step(self) -> None:
    patch = self._selector.select(candidates)
    if patch:
        version = self._registry.register(
            module_name=patch.module_name,
            implementation=patch.implementation,
            metadata={"source": "synthesis"},
        )
        result = await self._swapper.swap(
            SwapRequest(module_name=patch.module_name,
                        target_version=version.version)
        )
        match result.phase:
            case SwapPhase.COMMITTED:
                self._audit.append(AuditRecord(event_type=AuditEventType.PATCH_APPLIED, ...))
            case SwapPhase.ROLLED_BACK:
                logger.warning("Swap rolled back: %s", result.error)

---

Prometheus Observability

asi_hot_swapper_swaps_total{module, phase}    # Counter — by terminal phase
asi_hot_swapper_duration_seconds{module}      # Histogram — end-to-end latency
asi_hot_swapper_rollbacks_total{module}       # Counter — verify failures
asi_hot_swapper_concurrent                    # Gauge — active right now
asi_hot_swapper_timeout_total{module}         # Counter — timeout aborts

---

Open Questions

1. Canary swapping — should HotSwapper support routing X% of traffic to the new version before full commit?
2. health_check timeout — should the module control its own probe deadline or defer to SwapperConfig?
3. Multi-module atomic swaps — if two modules form a coupled pair, can we guarantee both swap or both rollback atomically?

Wiki: Phase-15-Hot-Swapper | Issue: #402