🔬 Show & Tell 25.5 — EmbodiedOrchestrator Architecture

[web3guru888]

🔬 Show & Tell 25.5 — EmbodiedOrchestrator Architecture

Overview

Phase 25.5 introduces EmbodiedOrchestrator, the capstone component that composes all Phase 25 modules into a unified perception-action loop. Grounded in George Lakoff's embodied cognition theory (1987) and Lawrence Barsalou's perceptual symbol systems (1999), the EmbodiedOrchestrator implements the thesis that cognition is fundamentally shaped by the body's interactions with the world.

Perception-Action Loop

┌─────────────────────────────────────────────────────────────────┐
│                    PERCEPTION-ACTION LOOP                        │
│                  (EmbodiedOrchestrator)                          │
│                                                                  │
│    ┌─────────┐     ┌───────────┐     ┌──────────┐              │
│    │ PERCEIVE│────▶│  DETECT   │────▶│  PLAN    │              │
│    │         │     │           │     │          │              │
│    │ Sensor  │     │ Affordance│     │ Motor    │              │
│    │ Fusion  │     │ Detector  │     │ Planner  │              │
│    │ (25.1)  │     │ (25.2)    │     │ (25.3)   │              │
│    └────▲────┘     └───────────┘     └────┬─────┘              │
│         │                                  │                     │
│         │          ┌───────────┐          │                     │
│         │          │ SIMULATE  │◀─────────┤                     │
│         │          │           │          │                     │
│         │          │ Forward   │          │                     │
│         │          │ Model     │          │                     │
│         │          │ (predict  │          │                     │
│         │          │  outcome) │          │                     │
│         │          └─────┬─────┘          │                     │
│         │                │ OK?            │                     │
│         │                ▼                ▼                     │
│    ┌────┴────┐     ┌───────────┐     ┌──────────┐              │
│    │ UPDATE  │◀────│  SPATIAL  │◀────│  ACT     │              │
│    │         │     │  CONTEXT  │     │          │              │
│    │ World   │     │           │     │ Execute  │              │
│    │ Model   │     │ Spatial   │     │ Motor    │              │
│    │ (13.1)  │     │ Reasoner  │     │ Plan     │              │
│    │         │     │ (25.4)    │     │          │              │
│    └─────────┘     └───────────┘     └──────────┘              │
│                                                                  │
│    Loop frequency: configurable (default 10 Hz)                 │
└─────────────────────────────────────────────────────────────────┘

Embodied Simulation Pipeline

Before executing a motor plan, the EmbodiedOrchestrator runs a forward model simulation to predict sensory consequences:

┌──────────────────────────────────────────────────────┐
│              EMBODIED SIMULATION                      │
│                                                       │
│  Input: MotorPlan + current FusedState + SpatialMap  │
│                                                       │
│  Step 1: Initialize simulated state = current state  │
│  Step 2: For each waypoint in MotorPlan:             │
│    ├─ Apply simulated motor command                   │
│    ├─ Forward dynamics: predict next state            │
│    ├─ Check: collision? joint limit? stability?       │
│    └─ Record: predicted sensory outcome               │
│                                                       │
│  Step 3: Compare predicted outcome vs goal           │
│    ├─ Match: proceed with execution                   │
│    ├─ Mismatch: re-plan with corrected model         │
│    └─ Dangerous: abort + safe state                   │
│                                                       │
│  Output: SimulationResult                             │
│    .predicted_states[]    — state at each step        │
│    .predicted_outcome     — final state               │
│    .confidence            — model confidence          │
│    .safety_check          — PASS/WARN/FAIL            │
│                                                       │
│  Compute bound: max_steps (default 100) + timeout    │
│  Partial: returns with low confidence if bounded     │
└──────────────────────────────────────────────────────┘

Grounded Language Understanding

Following Barsalou's Perceptual Symbol Systems, abstract concepts are grounded in sensorimotor experience:

┌─────────────────────────────────────────────────────────┐
│           GROUNDED LANGUAGE UNDERSTANDING                 │
│                                                          │
│  Input: linguistic concept (e.g., "heavy")               │
│                                                          │
│  ┌──────────────┐                                        │
│  │ Concept       │  "heavy"                              │
│  └──────┬───────┘                                        │
│         ▼                                                │
│  ┌──────────────────────────────┐                        │
│  │ Perceptual Symbol Lookup     │                        │
│  │ concept_map["heavy"] → {     │                        │
│  │   proprioceptive: high_force,│                        │
│  │   visual: large_volume,      │                        │
│  │   motor: slow_lift_trajectory│                        │
│  │ }                            │                        │
│  └──────────┬───────────────────┘                        │
│             ▼                                            │
│  ┌──────────────────────────────┐                        │
│  │ Sensorimotor Activation      │                        │
│  │ • Activate proprioceptive    │                        │
│  │   simulation (high force)    │                        │
│  │ • Activate visual expectation│                        │
│  │   (large dense object)       │                        │
│  │ • Activate motor program     │                        │
│  │   (careful, slow lift)       │                        │
│  └──────────┬───────────────────┘                        │
│             ▼                                            │
│  ┌──────────────────────────────┐                        │
│  │ EmbodiedContext              │                        │
│  │ .sensory_expectations[]      │                        │
│  │ .motor_programs[]            │                        │
│  │ .confidence                  │                        │
│  │ .grounding_source            │                        │
│  └──────────────────────────────┘                        │
│                                                          │
│  Used by: ResponseGenerator (19.3), DialogueManager (19.2)│
└─────────────────────────────────────────────────────────┘

Phase 25 Component Composition

┌──────────────────────────────────────────────────────────────┐
│               EmbodiedOrchestrator (25.5)                     │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐  │
│  │                                                        │  │
│  │   SensorFusion ──→ AffordanceDetector ──→ MotorPlanner │  │
│  │     (25.1)            (25.2)               (25.3)      │  │
│  │       │                  │                    │        │  │
│  │       └──────────────────┼────────────────────┘        │  │
│  │                          │                             │  │
│  │                    SpatialReasoner                      │  │
│  │                       (25.4)                           │  │
│  │                                                        │  │
│  └────────────────────────────────────────────────────────┘  │
│                                                              │
│  Composition contract:                                       │
│  1. fuse_sensors()      → FusedState                        │
│  2. detect_affordances() → Set[Affordance]                  │
│  3. plan_motion()        → MotorPlan                        │
│  4. simulate()           → SimulationResult                 │
│  5. execute()            → ActionOutcome                    │
│  6. update_spatial()     → SpatialMap                       │
│  7. update_world_model() → Beliefs                          │
│  8. → loop back to (1)                                      │
└──────────────────────────────────────────────────────────────┘

Prometheus Metrics

Metric	Type	Description
asi_embodied_loop_iterations_total	Counter	Perception-action loop iterations
asi_embodied_loop_latency_seconds	Histogram	Full loop cycle time
asi_embodied_simulation_runs_total	Counter	Forward model simulation runs
asi_embodied_simulation_safety_failures	Counter	Simulations that flagged safety issues
asi_embodied_grounding_lookups_total	Counter	Grounded language understanding lookups

PromQL:

# Loop frequency (should be ~10 Hz)
rate(asi_embodied_loop_iterations_total[1m])

# Safety check failure rate
rate(asi_embodied_simulation_safety_failures[5m]) / rate(asi_embodied_simulation_runs_total[5m])

# Grounding cache miss (new concepts)
rate(asi_embodied_grounding_lookups_total[5m])

Integration with Phases 19, 20, 24

Phase 19 (Communication):
  ResponseGenerator 19.3 ←── EmbodiedOrchestrator (grounded language)
  DialogueManager 19.2 ←── EmbodiedOrchestrator (embodied context for dialogue)

Phase 20 (Reasoning):
  AnalogicalReasoner 20.2 ←── EmbodiedOrchestrator (spatial analogies)
  LogicalInferenceEngine 20.1 ←── EmbodiedOrchestrator (physical constraints as axioms)

Phase 24 (Social Intelligence):
  SocialOrchestrator 24.5 ←── EmbodiedOrchestrator (gesture, proxemics, embodied social signals)
  PerspectiveTaker 24.3 ←── SpatialReasoner 25.4 (spatial perspective grounding)

Open Questions

1. How should the forward model be learned vs hand-coded? (Physics engine vs neural dynamics model)
2. What is the optimal loop frequency trade-off between responsiveness and compute cost?
3. How does grounded language understanding scale to abstract concepts (e.g., "justice", "freedom")?

📌 Spec: #566 | Q&A: See companion discussion