Q&A 39.4 — AttentionVisualizer: LRP Rules, Conservation & Multi-Head Analysis

[web3guru888]

AttentionVisualizer — Configuration & FAQ

Q: Does attention equal explanation?

A: No. Jain & Wallace (2019) showed that attention weights often do not correlate with gradient-based feature importance. Attention rollout (Abnar & Zuidema, 2020) partially addresses this by accounting for information mixing across layers. Always cross-validate attention-based explanations with FeatureAttributor (39.1) results.

Q: Which LRP rule should I use?

A:

Rule	Best For	Behavior
epsilon-rule	General use	Absorbs small, contradictory contributions
gamma-rule	Positive evidence emphasis	Favors positive over negative contributions
alpha-beta rule	Fine-grained control	alpha=1, beta=0 for positive-only relevance
Deep Taylor	Theoretical rigor	First-order Taylor decomposition per layer

Start with epsilon-rule (epsilon=0.01). Increase epsilon for smoother, sparser explanations.

Q: How to handle multi-head attention with many heads?

A: For models with 12-96 heads: (1) Use head pruning to identify the most informative heads (Michel et al. 2019), (2) Cluster heads by their attention patterns (same-behavior groups), (3) Show top-K heads ranked by gradient-weighted importance rather than all heads.

Q: How to verify LRP conservation?

A: Check that total input relevance equals the output prediction: |Sum_i R_i - f(x)| / |f(x)| < threshold (typically 1e-4). Violations indicate numerical issues or unsupported layer types. The conservation_error field in RelevanceMap tracks this automatically.

Q: Can GradCAM work for text models (not just vision)?

A: Yes, with adaptations: (1) Use the last transformer block as the target layer, (2) Pool GradCAM scores over the hidden dimension to get per-token scores, (3) For encoder-decoder models, compute GradCAM separately for encoder and decoder. The token-level GradCAM scores correlate well with integrated gradients (Bastings & Filippova, 2020).

Q: What is the computational overhead of attention visualization?

Method	Extra Forward	Extra Backward	Memory
Attention extraction	0	0	O(LHS^2)
Rollout	0	0	O(S^2)
Gradient rollout	0	1	O(LHS^2)
LRP	0	L (per layer)	O(model_params)
GradCAM	0	1	O(hidden_dim*S)

Related: #791