Show & Tell 39.2 — ConceptExplainer: TCAV, Concept Bottleneck Models & Network Dissection

[web3guru888]

ConceptExplainer — Algorithm Deep Dive

TCAV (Testing with Concept Activation Vectors)

Kim et al. (2018) introduced TCAV to answer: "How important is concept C to the prediction of class K?"

Algorithm: TCAV
Input: model f, concept C (positive/negative examples), target class k, layer l
Output: TCAV score ∈ [0, 1]

1. Extract activations at layer l for positive and negative concept examples
2. Train linear classifier (SVM/logistic) to distinguish C+ from C-
3. CAV = normal vector to the decision boundary (unit vector v_C^l)
4. For each test instance x of class k:
   a. Compute gradient of class logit: g = ∂f_k/∂a_l(x)
   b. Compute conceptual sensitivity: S_C(x) = g · v_C^l
5. TCAV_score = |{x : S_C(x) > 0}| / |test_set|
6. Statistical test: repeat with random concepts, p-value via permutation

Key insight: A TCAV score of 0.8 for concept "striped" and class "zebra" means 80% of zebra images have positive conceptual sensitivity to "striped."

Concept Bottleneck Models (Koh et al. 2020)

Standard model:     x → f(x) → ŷ  (black-box)
Concept bottleneck: x → g(x) → ĉ → h(ĉ) → ŷ  (interpretable)

Where:
  g: input → concept predictions (ĉ₁, ĉ₂, ..., ĉₘ)
  h: concept values → final prediction
  Each ĉᵢ has human-interpretable meaning

Benefits: (1) Inherently interpretable — inspect concept activations, (2) Human intervention — experts can correct concept predictions at test time, (3) Modular — swap concept predictors independently.

Prototype-Based Explanations

Algorithm: MMD-critic Prototype Selection
Input: dataset X, kernel K, n_prototypes p, n_criticisms c
Output: prototypes P, criticisms C

1. Greedily select prototypes to maximize MMD:
   P = argmax_{|P|=p} MMD²(P, X)
   where MMD²(P, X) = (2/p)Σᵢ E_x[K(pᵢ,x)] - (1/p²)Σᵢⱼ K(pᵢ,pⱼ)
2. Select criticisms — instances poorly represented by prototypes:
   C = argmax_{|C|=c} Σⱼ (1/|X|)Σₓ K(cⱼ,x) - (1/p)Σᵢ K(cⱼ,pᵢ)

Integration with Existing Phases

• Phase 28 (Attention): Map concepts to attention patterns — which heads attend to which concepts?
• Phase 26 (Knowledge): ConceptGraph nodes become TCAV concept definitions
• Phase 22 (Creative Intelligence): Explain creative outputs through concept composition

See issue #789 for full spec.