Q&A: Phase 9.3 FederatedTaskRouter — routing strategies, load_score, stale_cap_ms, capabilities, custom strategies, and Grafana monitoring

[web3guru888]

Q&A — FederatedTaskRouter configuration and behaviour

Answers to the most common questions about the Phase 9.3 FederatedTaskRouter.

---

Q1: Which routing strategy should I use by default?

Use capability_first unless you have a specific reason to switch.

Strategy	When to use
capability_first	Default — heterogeneous cluster, tasks need specific module capabilities
consistent_hash	Uniform cluster, you want cache/session affinity per task type
auction	High-throughput, latency-sensitive; peers have measurable latency differences

---

Q2: How is load_score measured and updated?

load_score (0.0–1.0) is written by each node to FederatedBlackboard under the key "peer/{peer_id}/cap" during its heartbeat loop (typically every 2 s, same interval as FederationGateway health poll).

load_score	Interpretation
0.0 – 0.3	Idle / lightly loaded
0.3 – 0.7	Moderate load
0.7 – 0.9	Heavy load — strategy may prefer other peers
≥ 0.9	Saturated — ConsistentHashStrategy skips this peer entirely

A simple implementation: load_score = active_cycles / max_concurrent_cycles.

---

Q3: What does stale_cap_ms control?

stale_cap_ms (default 5 000 ms) is the maximum age of a PeerCapSnapshot before it is ignored. If a peer has not written a fresh "peer/{peer_id}/cap" entry to the FederatedBlackboard within this window, its snapshot is discarded and asi_router_stale_snapshots_total is incremented.

Tune this to slightly above your heartbeat interval:

Heartbeat interval	Recommended stale_cap_ms
1 s	3 000 ms
2 s	5 000 ms (default)
5 s	10 000 ms

---

Q4: What happens when required_caps is an empty frozenset?

An empty required_caps means "any peer will do." All strategies treat every candidate as eligible.

• CapabilityFirstStrategy → picks the least-loaded peer
• ConsistentHashStrategy → picks via rendezvous hash (ignoring caps)
• AuctionStrategy → picks by lowest bid score

---

Q5: How do I register my node's capabilities?

Write a PeerCapSnapshot to the FederatedBlackboard in your startup or heartbeat loop:

import json, time

snapshot = PeerCapSnapshot(
    peer_id      = my_gateway.local_peer_id,
    capabilities = frozenset({"reasoning", "kv_store", "vector_search"}),
    load_score   = my_load_meter.score(),
    latency_ms   = 0.0,       # will be measured by peers on first contact
    updated_at_ms = int(time.time() * 1000),
)
await blackboard.write(
    key    = f"peer/{snapshot.peer_id}/cap",
    value  = json.dumps(snapshot.__dict__).encode(),
    author = snapshot.peer_id,
)

---

Q6: How do I add a custom RoutingStrategy?

Implement the RoutingStrategy Protocol — any class with a select(envelope, candidates) -> str | None method qualifies:

class PriorityFirstStrategy:
    """Always route highest-priority tasks to the fastest (lowest-latency) peer."""
    def select(self, envelope: TaskEnvelope, candidates: list[PeerCapSnapshot]) -> str | None:
        if not candidates:
            return None
        if envelope.priority >= 10:
            return min(candidates, key=lambda p: p.latency_ms).peer_id
        return min(candidates, key=lambda p: p.load_score).peer_id

Pass it to build_task_router(strategy=PriorityFirstStrategy()).

---

Q7: How do I monitor routing health with Grafana?

Recommended dashboard panels:

# Routing outcome distribution (stacked bar)
sum by (decision) (rate(asi_router_tasks_total[5m]))

# P95 routing latency by decision
histogram_quantile(0.95,
  sum by (le, decision) (rate(asi_router_latency_seconds_bucket[5m]))
)

# Retry rate (spikes = peer instability)
rate(asi_router_retries_total[5m])

# Stale snapshot rate (spikes = heartbeat gaps)
rate(asi_router_stale_snapshots_total[5m])

# Active peers (should match known cluster size)
asi_router_active_peers

Alert rule — fallback surge:

alert: HighFallbackRate
expr: rate(asi_router_tasks_total{decision="fallback"}[5m]) > 0.1
for: 2m
annotations:
  summary: "Router falling back to local execution — check peer health"