Q&A: Phase 18.4 — CausalMemoryIndex Design Questions

[web3guru888]

Phase 18.4 — CausalMemoryIndex: Q&A

Common questions about the CausalMemoryIndex design (spec: #459).

---

Q1: Why four IndexModes instead of a single generic query() method?

A: Each mode uses a fundamentally different data structure and algorithm:

Mode	Structure	Algorithm
CAUSE_CHAIN	_cause_adj dict	BFS backward
EFFECT_FAN	_effect_adj dict	BFS forward
TEMPORAL_RANGE	SortedList	bisect + irange
SALIENCE_TOP_K	_entries values	heapq.nlargest + decay

A generic query(mode, **kwargs) would require runtime dispatch with untyped kwargs — losing type safety and making mypy unhappy. Separate methods give each query a precise signature and enable the LRU cache to key on exact parameter tuples.

---

Q2: How does the salience decay formula work, and why exponential?

A: The formula is:

decayed_salience = raw_salience × exp(−λ × age_seconds)

Exponential decay is standard in memory models (Ebbinghaus forgetting curve). Key properties:

• Monotonically decreasing: older memories always score lower, all else equal
• Smooth: no sudden cliffs — the score degrades gradually
• Tuneable: a single parameter λ (decay_rate) controls the half-life
• Composable: if a memory is "refreshed" (re-consolidated), its timestamp_ns resets, effectively boosting its decayed salience back up

With λ = 1e-6, half-life ≈ ln(2) / 1e-6 ≈ 693,147 seconds ≈ 8 days. This is deliberately slow — important memories should persist for about a week before dropping to 50%.

---

Q3: Why SortedList instead of a B-tree or database index?

A: sortedcontainers.SortedList is a pure-Python sorted list backed by a list-of-lists structure that gives:

• O(log n) insert/remove
• O(log n + k) range queries via irange()
• Zero external dependencies (no C extensions, no database process)
• Battle-tested in production Python (widely used in competitive programming and production systems)

A B-tree (e.g., via blist or SQLite) would add complexity for marginal gain at our expected scale (< 100K entries). If the index grows beyond 1M entries, we'd consider moving the temporal index to a proper B+ tree or an embedded database like LMDB.

---

Q4: How does cache invalidation work when index_chunk is called?

A: On every index_chunk(entry), the cache invalidation follows this algorithm:

1. Compute the "blast radius": affected = {entry.chunk_id} ∪ entry.cause_ids ∪ entry.effect_ids
2. Iterate over all cache entries
3. For each cached result set, check if any result's chunk_id is in affected
4. If yes, evict that cache entry

Additionally, entries expire after cache_ttl_s (default 60s) — so even without explicit invalidation, stale results are bounded.

The cache is intentionally simple (OrderedDict with TTL) rather than a full tag-based invalidation system. At typical cache sizes (< 100 entries due to TTL), the O(n) scan on invalidation takes < 1ms.

---

Q5: How does CausalMemoryIndex integrate with CausalGraph (8.2)?

A: CausalMemoryIndex does not import or depend on CausalGraph at runtime for queries. Instead:

1. At rebuild time: rebuild() calls an injected callback (or Protocol method) that fetches all edges from CausalGraph. These edges are projected into the local _cause_adj and _effect_adj dicts, filtered to only include chunk_ids present in _entries.

2. At index time: When index_chunk() is called, the cause_ids and effect_ids in the IndexEntry are assumed to have been resolved from CausalGraph by the caller (typically MemoryConsolidator or a pipeline stage).

This loose coupling avoids import cycles and allows testing CausalMemoryIndex with mock data without standing up a full CausalGraph instance.

---

Q6: How should the rebuild interval be tuned?

A: The default rebuild_interval_s = 300 (5 minutes) balances freshness vs CPU cost:

Workload	Recommended interval	Rationale
Low-traffic dev	600–900s	Fewer edges change; save CPU
Normal production	300s (default)	Good balance
High-throughput	60–120s	Many concurrent edge additions
Real-time critical	30s + incremental	Rely on index_chunk for freshness, rebuild as safety net

Monitor asi_causal_index_rebuild_duration_seconds — if rebuilds take > 50% of the interval, increase the interval or optimise the upstream data pull.

---

Q7: What is the embedding field for, and when will it be used?

A: The embedding: tuple[float, ...] | None field on IndexEntry is a forward-looking extension point for semantic similarity queries.

Current plan:

• Phase 18.4: Field exists but is always None. No query mode uses it.
• Future (post-18): A SEMANTIC_KNN IndexMode could be added that uses approximate nearest-neighbour search (e.g., HNSW via hnswlib or faiss) over the embedding vectors.

The field is included now so that:

1. The IndexEntry schema doesn't need a breaking change later
2. Callers that already have embeddings (e.g., from a language model) can start populating the field
3. The frozen dataclass stays forward-compatible

Since embedding defaults to None and is a tuple (immutable, hashable), it doesn't affect memory usage or performance for entries that don't use it.