Phase 8.4 — Docker/Helm: containerisation & Kubernetes deployment for ASI-Build

[web3guru888]

Overview

Phase 8.4 delivers production-grade containerisation and Kubernetes deployment manifests for the full ASI-Build stack (cognitive core + ExplainAPI + Prometheus/Grafana). It is the first deployment-hardening sub-phase of Phase 8 and lays the groundwork for Phase 8.5 (Sepolia CI).

---

Motivation

The ASI-Build runtime today lives exclusively in Python source files. Shipping it to a real server or cloud cluster requires:

Gap	Solution
No reproducible runtime environment	Multi-stage Dockerfile with pinned base image
Local dev friction (seven services)	docker-compose.yml one-command stack
No auto-scaling / self-healing	Helm chart with HPA + liveness/readiness probes
No in-cluster metrics scraping	Prometheus ServiceMonitor CRD via kube-prometheus-stack
No resource quotas	resources: limits in every Deployment

---

Deliverables

1. Dockerfile (multi-stage)

Stage 0  builder  — python:3.11-slim, install all deps, compile *.pyc
Stage 1  runtime  — python:3.11-slim, copy site-packages + app, drop to non-root uid=1000

• Build arg ASI_VERSION baked into image label
• HEALTHCHECK calls GET /health on the ExplainAPI port (8080)
• Final image ≤ 350 MB (no dev tools, no pip cache)

2. docker-compose.yml (local dev / CI)

Services:

Service	Image	Port
asi-core	./Dockerfile	—
explain-api	./Dockerfile	8080
prometheus	prom/prometheus:v2.51	9090
grafana	grafana/grafana:10.4	3000
redis	redis:7-alpine	6379

Volume mounts: ./config/prometheus.yml, ./config/grafana/, ./data/traces/

3. Helm chart (charts/asi-build/)

charts/asi-build/
├── Chart.yaml          # apiVersion: v2, appVersion from ASI_VERSION
├── values.yaml         # image.tag, replicaCount, resources, autoscaling
├── templates/
│   ├── deployment.yaml       # asi-core + explain-api containers
│   ├── service.yaml          # ClusterIP for explain-api
│   ├── ingress.yaml          # optional nginx ingress
│   ├── hpa.yaml              # HorizontalPodAutoscaler (CPU 70%)
│   ├── configmap.yaml        # prometheus.yml + grafana dashboards
│   ├── servicemonitor.yaml   # Prometheus Operator ServiceMonitor
│   └── _helpers.tpl

Key values.yaml knobs:

image:
  repository: ghcr.io/web3guru888/asi-build
  tag: latest
  pullPolicy: IfNotPresent

replicaCount: 2

resources:
  requests: { cpu: 500m, memory: 512Mi }
  limits:   { cpu: 2,    memory: 2Gi  }

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 8
  targetCPUUtilizationPercentage: 70

explainApi:
  port: 8080
  rateLimitPerMinute: 60

prometheus:
  serviceMonitor:
    enabled: true
    interval: 15s

4. Makefile targets

Target	Command
make docker-build	docker build --build-arg ASI_VERSION=$(VERSION) -t asi-build:$(VERSION) .
make docker-push	push to GHCR
make compose-up	docker compose up -d
make compose-down	docker compose down -v
make helm-lint	helm lint charts/asi-build
make helm-install	helm upgrade --install asi-build charts/asi-build -f values.yaml
make helm-template	render manifests to stdout

5. GitHub Actions workflow (.github/workflows/docker.yml)

Triggers: push to main, pull_request

Steps:

1. docker/setup-buildx-action
2. docker/login-action → GHCR
3. docker/build-push-action (multi-platform: linux/amd64 + linux/arm64)
4. helm/kind-action → spin up kind cluster
5. helm lint + helm install --dry-run
6. docker scout cves → fail on CRITICAL CVEs

---

Acceptance Criteria

• docker build . succeeds with no warnings
• docker compose up raises all 5 services; /health returns 200
• helm lint charts/asi-build exits 0
• helm install --dry-run renders valid YAML
• HPA manifest validates against Kubernetes 1.28 schema
• ServiceMonitor CRD recognised by kube-prometheus-stack 0.71
• GitHub Actions workflow passes on a clean runner
• Image size ≤ 350 MB (checked via docker image inspect)
• No CRITICAL CVEs in docker scout scan
• All 12 test targets pass inside the container (pytest -x)

---

Implementation Order

1. Write Dockerfile (multi-stage, non-root, HEALTHCHECK)
2. Write docker-compose.yml with named volumes + healthchecks
3. Write config/prometheus.yml scrape config (all 5 job names)
4. Write charts/asi-build/Chart.yaml + values.yaml
5. Write templates/deployment.yaml (liveness/readiness on /health)
6. Write templates/hpa.yaml + templates/servicemonitor.yaml
7. Write templates/configmap.yaml (Grafana dashboard JSON)
8. Write Makefile with all 7 targets
9. Write .github/workflows/docker.yml
10. Run helm lint + docker build locally, fix errors
11. Open PR, CI green, merge

---

Phase 8 Sub-phase Tracker

Sub-phase	Issue	Status
8.1 DecisionTracer	#276	✅ spec complete
8.2 CausalGraph	#280	✅ spec complete
8.3 ExplainAPI	#283	✅ spec complete
8.4 Docker/Helm	#291	🟡 in progress
8.5 Sepolia CI	TBD	📋 planned

---

Related

• Phase 8.3 ExplainAPI: Phase 8.3 — ExplainAPI: HTTP introspection layer for DecisionTracer & CausalGraph #283 (the HTTP service being containerised)
• Phase 8.5 Sepolia CI: planned (will depend on images built here)
• Discussion Ideas: Phase 8 remaining — 8.4 Docker/Helm hardening vs 8.5 Sepolia CI pipeline #290 (planning): Docker/Helm was the recommended first step

[web3guru888]

Implementation Notes

Dockerfile — multi-stage sketch

# syntax=docker/dockerfile:1.7
ARG PYTHON_VERSION=3.11-slim
ARG ASI_VERSION=dev

# ── Stage 0: builder ──────────────────────────────────────────────────────────
FROM python:${PYTHON_VERSION} AS builder
WORKDIR /build

COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

COPY . .
# Pre-compile to .pyc so runtime import is faster
RUN python -m compileall -q asi/

# ── Stage 1: runtime ──────────────────────────────────────────────────────────
FROM python:${PYTHON_VERSION} AS runtime
ARG ASI_VERSION
LABEL org.opencontainers.image.version=${ASI_VERSION}       org.opencontainers.image.source="https://github.com/web3guru888/asi-build"

# Non-root user
RUN useradd -u 1000 -m asi
WORKDIR /app

COPY --from=builder /install /usr/local
COPY --from=builder /build/asi ./asi
COPY --from=builder /build/config ./config

USER asi

EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3   CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8080/health')"

CMD ["python", "-m", "uvicorn", "asi.explain_api.app:app", "--host", "0.0.0.0", "--port", "8080"]

Key points:

• keeps site-packages separate → easy COPY in stage 1
• (uid 1000) — never run as root in production
• uses stdlib only (no curl/wget in slim image)
• ARG flows through both stages via OCI label

---

docker-compose.yml — healthcheck pattern

services:
  explain-api:
    build:
      context: .
      args:
        ASI_VERSION: ${ASI_VERSION:-dev}
    ports: ["8080:8080"]
    healthcheck:
      test: ["CMD", "python", "-c",
             "import urllib.request; urllib.request.urlopen('http://localhost:8080/health')"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 15s
    depends_on:
      redis:
        condition: service_healthy

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      retries: 5

Use on every dependency so services start in the right order.

---

Helm HPA — autoscaling/v2 (Kubernetes ≥ 1.23)

# templates/hpa.yaml
{{- if .Values.autoscaling.enabled }}
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: {{ include "asi-build.fullname" . }}
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: {{ include "asi-build.fullname" . }}
  minReplicas: {{ .Values.autoscaling.minReplicas }}
  maxReplicas: {{ .Values.autoscaling.maxReplicas }}
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
{{- end }}

Use (not ) — it is stable since Kubernetes 1.26.

---

ServiceMonitor — kube-prometheus-stack

# templates/servicemonitor.yaml
{{- if .Values.prometheus.serviceMonitor.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: {{ include "asi-build.fullname" . }}
  labels:
    release: kube-prometheus-stack   # must match operator's serviceMonitorSelector
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: {{ include "asi-build.name" . }}
  endpoints:
    - port: metrics
      interval: {{ .Values.prometheus.serviceMonitor.interval }}
      path: /metrics
{{- end }}

The label is required — the Prometheus Operator won't pick up ServiceMonitors without it matching .

---

prometheus.yml — scrape config for all 5 metric families

scrape_configs:
  - job_name: explain_api
    static_configs:
      - targets: ["explain-api:8080"]
  - job_name: causal_graph
    static_configs:
      - targets: ["asi-core:9100"]
  - job_name: decision_tracer
    static_configs:
      - targets: ["asi-core:9100"]
  - job_name: sleep_orchestrator
    static_configs:
      - targets: ["asi-core:9100"]
  - job_name: replay_buffer
    static_configs:
      - targets: ["asi-core:9100"]

In a Kubernetes deployment, replace with pointing to the Service port.

---

Mypy / type-checking

File	Key types
Dockerfile	N/A
docker-compose.yml	N/A
charts/**/*.yaml	helm lint validates
.github/workflows/docker.yml	actionlint validates

Run and in CI.

---

Test targets (12)

1. exits 0
2. → size ≤ 350 MB
3. → all pass
4. → valid YAML
5. → 200
6. → all services healthy
7. exits 0
8. → valid YAML
9. → no errors
10. HPA / respected (kind cluster test)
11. ServiceMonitor discovered by Prometheus Operator (kind + kube-prometheus-stack)
12. → 0 CRITICAL CVEs

---

10-step implementation order

1. Create (pinned, generated)
2. Write +
3. Write +
4. • locally — fix any issues
5. Scaffold with ; replace generated templates
6. Write with liveness/readiness probes
7. Write +
8. Write (prometheus.yml + Grafana dashboard JSON)
9. Write +
10. • • — iterate until clean

[web3guru888]

Phase 8.5 — Sepolia CI filed

Issue #295 — Phase 8.5 — Sepolia CI: contract redeployment, bridge health checks, and on-chain fuzz testing

Phase 8.4 (Docker/Helm) → Phase 8.5 (Sepolia CI) integration point: the sepolia-exporter service is added to docker-compose.yml and the Helm chart as part of Phase 8.5, extending the Phase 8.4 deployment manifests.

Phase 8 sub-phase tracker:

Sub-phase	Issue	Status
8.1 DecisionTracer	#276	✅ spec complete
8.2 CausalGraph	#280	✅ spec complete
8.3 ExplainAPI	#283	✅ spec complete
8.4 Docker/Helm	#291	✅ spec complete
8.5 Sepolia CI	#295	🟡 in progress

Phase 8 is now fully spec'd across all 5 sub-phases.