Show & Tell — Phase 14.2: SandboxRunner architecture and execution backends

[web3guru888]

SandboxRunner — Component Map

 CognitiveCycle
       │
       ▼
 CodeSynthesiser.synthesise(sandbox=runner)
       │
       ▼
 SandboxRunner.run(ExecutionRequest)
       │
       ├── SUBPROCESS ─► asyncio.create_subprocess_exec
       │                        + asyncio.wait_for (timeout)
       │                        + preexec_fn (resource limits)
       │
       ├── DOCKER ──────► docker run --rm --memory --cpus
       │
       ├── WASM ────────► wasmtime / wasmer
       │
       └── NSJAIL ──────► nsjail (Linux namespaces + seccomp)

---

Execution Backend Comparison

Backend	Isolation	Deps	Overhead	Best for
SUBPROCESS	resource ulimits + OS process	stdlib only	~2ms	Dev / CI — Python/JS/Bash
DOCKER	Full container namespace	docker SDK	~500ms cold	Production multi-language
WASM	WASM sandbox, no syscalls	wasmtime binary	~50ms	Untrusted WASM modules
NSJAIL	Linux namespace + seccomp + cgroup	nsjail binary	~20ms	High-security Linux prod

---

Dataclass Hierarchy

ResourceLimits
  max_memory_mb: int = 256
  max_cpu_s: float = 10.0
  max_open_files: int = 64

ExecutionRequest
  code: str
  language: str
  timeout_s: float = 30.0
  resource_limits: ResourceLimits
  stdin: str | None = None

ExecutionResult
  stdout: str
  stderr: str
  exit_code: int
  elapsed_s: float
  timed_out: bool
  backend: ExecutionBackend
  memory_exceeded: bool = False

SandboxConfig
  backend: ExecutionBackend = SUBPROCESS
  default_timeout_s: float = 30.0
  resource_limits: ResourceLimits
  docker_image: str = "python:3.12-slim"
  wasm_engine: str = "wasmtime"
  nsjail_path: str = "/usr/sbin/nsjail"

---

preexec_fn — Resource Limits via resource stdlib

import resource

def _make_preexec(limits: ResourceLimits):
    def _set():
        # Address space ceiling
        mem = limits.max_memory_mb * 1024 * 1024
        resource.setrlimit(resource.RLIMIT_AS, (mem, mem))
        # CPU time ceiling
        resource.setrlimit(resource.RLIMIT_CPU,
                           (int(limits.max_cpu_s)+1, int(limits.max_cpu_s)+1))
        # Open file descriptor ceiling
        resource.setrlimit(resource.RLIMIT_NOFILE,
                           (limits.max_open_files, limits.max_open_files))
    return _set

preexec_fn runs in the child process after fork() but before exec(), so it cannot affect the parent event loop.

---

Timeout Enforcement Pattern

try:
    proc = await asyncio.create_subprocess_exec(
        interp, tmp_path,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
        preexec_fn=_make_preexec(request.resource_limits),
    )
    stdout_b, stderr_b = await asyncio.wait_for(
        proc.communicate(), timeout=request.timeout_s
    )
except asyncio.TimeoutError:
    proc.kill()           # SIGKILL — no grace period
    await proc.wait()     # reap zombie
    return ExecutionResult(
        stdout="", stderr="Execution timed out",
        exit_code=-1, elapsed_s=elapsed,
        timed_out=True, backend=backend,
    )

---

batch_run — Concurrency with Partial Failure

async def batch_run(
    self, requests: list[ExecutionRequest]
) -> list[ExecutionResult]:
    raw = await asyncio.gather(
        *[self.run(r) for r in requests],
        return_exceptions=True,
    )
    return [
        r if isinstance(r, ExecutionResult)
        else ExecutionResult(
            stdout="", stderr=str(r), exit_code=-1,
            elapsed_s=0.0, timed_out=False,
            backend=self._config.backend,
        )
        for r in raw
    ]

All requests run concurrently — a single failure is coerced into an ExecutionResult with exit_code=-1, so the rest are unaffected.

---

Prometheus Metrics

Metric	Type	Labels
sandbox_runs_total	Counter	backend, language
sandbox_timeouts_total	Counter	backend
sandbox_errors_total	Counter	backend
sandbox_elapsed_seconds	Histogram	backend
sandbox_memory_exceeded_total	Counter	backend

PromQL — timeout rate by backend:

rate(sandbox_timeouts_total[5m])
/ rate(sandbox_runs_total[5m])

PromQL — p95 latency:

histogram_quantile(0.95, rate(sandbox_elapsed_seconds_bucket[5m]))

Grafana alert:

- alert: SandboxHighTimeoutRate
  expr: >
    rate(sandbox_timeouts_total[5m])
    / rate(sandbox_runs_total[5m]) > 0.10
  for: 2m
  labels:
    severity: warning
  annotations:
    summary: "Sandbox timeout rate >10% on {{ $labels.backend }}"

---

Open Questions

1. Windows support — preexec_fn is POSIX-only. Should we add a Windows fallback using subprocess.CREATE_NEW_PROCESS_GROUP + ctypes Job Objects?
2. Docker layer caching — For the DOCKER backend, should we pre-pull images during startup to avoid cold-start penalty?
3. Multi-language WASM — How should we handle languages that don't compile to WASM natively (e.g., Python via Pyodide)?

Discussion thread for Issue #388.