Phase 13.3 — CuriosityModule configuration Q&A

[web3guru888]

CuriosityModule — Configuration Q&A

Frequently asked questions about configuring and integrating CuriosityModule (Phase 13.3).

---

Q1: Which normalisation strategy should I use for a new environment?

Start with the default RUNNING (Welford online mean/variance). It requires no hyperparameters and adapts automatically as the world model improves. Switch to PERCENTILE if your surprise distribution is heavy-tailed (outlier transitions dominate the running std). Use TANH if you need a guaranteed bounded output for policy-gradient methods.

Strategy	When to use
RUNNING	General default — zero extra config
PERCENTILE	Heavy-tailed surprise; set percentile_lo/percentile_hi
TANH	Policy-gradient agents; smooth gradient flow
NONE	Ablation or when you pre-normalise externally

---

Q2: How fast does the intrinsic weight decay?

With the default EXPONENTIAL schedule at decay_rate=1e-5 and intrinsic_weight_init=0.1:

• At step 0: β ≈ 0.100
• At step 69,315 (half-life): β ≈ 0.050
• At step 200,000: β ≈ 0.014
• At step 460,517: β ≈ 0.001 (= intrinsic_weight_min, floor kicks in)

Use COSINE if you want a smoother ramp-down over a fixed training budget.

---

Q3: What happens when reset() is called?

reset() clears all Welford accumulators (_n, _mean, _M2) and the percentile buffer, and resets _weight to intrinsic_weight_init. Call it at the start of each episode for episodic novelty tracking. Omit reset() entirely for life-long curiosity that treats all past experience.

---

Q4: How do I wire CuriosityModule into CognitiveCycle?

# In CognitiveCycle.__init__:
from asi.curiosity import build_curiosity_module, CuriosityConfig
self._curiosity = build_curiosity_module(self._world_model, CuriosityConfig())

# In CognitiveCycle._step():
async def _step(self, obs, action, next_obs, extrinsic_reward, step):
    event = await self._curiosity.compute_bonus(obs, action, next_obs, step)
    blended = extrinsic_reward + event.bonus
    # pass blended reward to learner …
    return blended

---

Q5: Can CuriosityModule be disabled in production?

Yes — set enabled=False in CuriosityConfig. The module then returns a SurpriseEvent with bonus=0.0 without calling WorldModel.surprise(), so there is zero runtime overhead.

cfg = CuriosityConfig(enabled=False)
curiosity = build_curiosity_module(world_model, cfg)

---

Q6: How does batch_bonus() interact with the asyncio lock?

Each compute_bonus() call acquires _lock only for the short stat-update section (not during the WorldModel.surprise() await). This means multiple concurrent batch_bonus() calls can issue all surprise() requests in parallel — the lock is only held long enough to update Welford accumulators, which is microseconds.

# Safe to call concurrently from multiple tasks:
events = await curiosity.batch_bonus(transitions, step=global_step)

---

Q7: How do I add a Grafana panel for curiosity weight decay?

# Grafana alert — curiosity weight not decaying
- alert: CuriosityWeightStuck
  expr: asi_curiosity_weight > 0.09
  for: 10m
  labels:
    severity: warning
  annotations:
    summary: "CuriosityModule weight not decaying — check decay_rate config"
    dashboard: "ASI-Build / Phase 13"

Recommended panel layout:

• Row 1: asi_curiosity_weight (Gauge, time-series) — watch the exponential decay
• Row 2: histogram_quantile(0.95, rate(asi_curiosity_bonus_bucket[5m])) — p95 bonus
• Row 3: histogram_quantile(0.50, rate(asi_curiosity_raw_surprise_bucket[5m])) — median raw MSE
• Row 4: asi_curiosity_resets_total (counter) — episode resets