🧠 Show & Tell 22.1 — DivergentGenerator: divergent thinking architecture & genetic idea evolution

[web3guru888]

🧠 Phase 22.1 — DivergentGenerator: Architecture & Design

What is the DivergentGenerator?

The DivergentGenerator is the creativity foundation for Phase 22 (Creative Problem-Solving & Innovation Engine). It implements five distinct divergent-thinking strategies wrapped in a genetic algorithm that evolves idea populations toward maximum novelty and diversity.

Why divergent thinking first? Research on creative cognition (Guilford, 1967; Runco & Acar, 2012) consistently shows that ideational fluency (generating many ideas) and ideational flexibility (spanning many categories) are the strongest predictors of creative output quality. The DivergentGenerator maximises both before any convergent filtering occurs.

---

Architecture Overview

                    ┌─────────────────────┐
                    │     User Prompt      │
                    └──────────┬──────────┘
                               │
                    ┌──────────▼──────────┐
                    │   Strategy Router    │
                    │   (_dispatch)        │
                    └──┬──┬──┬──┬──┬──────┘
                       │  │  │  │  │
          ┌────────────┘  │  │  │  └────────────┐
          ▼               ▼  │  ▼               ▼
    ┌──────────┐  ┌────────┐ │ ┌──────────┐ ┌──────────────┐
    │ RANDOM   │  │SCAMPER │ │ │CONSTRAIN │ │MORPHOLOGICAL │
    │ COMBINE  │  │(7 ops) │ │ │RELAXATION│ │ANALYSIS      │
    └────┬─────┘  └───┬────┘ │ └────┬─────┘ └─────┬────────┘
         │            │      │      │              │
         │     ┌──────▼──┐   │      │              │
         │     │BISOCIATE │   │      │              │
         │     │(Koestler)│   │      │              │
         │     └────┬─────┘   │      │              │
         │          │         │      │              │
         └────┬─────┴────┬────┴──────┴──────┬──────┘
              │          │                  │
              ▼          ▼                  ▼
    ┌─────────────────────────────────────────────┐
    │         Raw Idea Pool (IdeaQuality.RAW)     │
    └──────────────────┬──────────────────────────┘
                       │ score_novelty()
                       ▼
    ┌─────────────────────────────────────────────┐
    │     Scored Population (novelty 0.0–1.0)     │
    │     ┌───────────────────────────────┐       │
    │     │ novelty = 0.6·cos_dist(centroid)      │
    │     │         + 0.4·min_nn_distance │       │
    │     └───────────────────────────────┘       │
    └──────────────────┬──────────────────────────┘
                       │ threshold filter (≥ 0.3)
                       ▼
    ┌─────────────────────────────────────────────┐
    │      Filtered Ideas (IdeaQuality.FILTERED)  │
    └──────────────────┬──────────────────────────┘
                       │
            ┌──────────▼──────────┐
            │  Genetic Algorithm  │
            │  evolve_population()│
            │                     │
            │  ┌───────────────┐  │
            │  │ TOURNAMENT    │  │
            │  │ SELECT (k=3)  │  │
            │  └───────┬───────┘  │
            │          │          │
            │  ┌───────▼───────┐  │
            │  │ CROSSOVER     │  │
            │  │ (rate=0.7)    │  │
            │  └───────┬───────┘  │
            │          │          │
            │  ┌───────▼───────┐  │
            │  │ MUTATE        │  │
            │  │ (rate=0.15)   │  │
            │  └───────┬───────┘  │
            │          │          │
            │  ┌───────▼───────┐  │
            │  │ SCORE+FILTER  │──┤ loop max_generations
            │  └───────────────┘  │
            └──────────┬──────────┘
                       │
                       ▼
    ┌─────────────────────────────────────────────┐
    │  Final Population (sorted by novelty desc)  │
    │  → ConvergentEvaluator (22.2)               │
    │  → CreativeBlender (22.3)                   │
    └─────────────────────────────────────────────┘

---

Strategy Deep-Dive

1. SCAMPER (Eberle, 1971)

Seven systematic operators applied to the prompt concept:

Operator	Transformation	Metadata Key
Substitute	Replace component X with Y	scamper_op: "substitute"
Combine	Merge concept A + concept B	scamper_op: "combine"
Adapt	Borrow pattern from domain D	scamper_op: "adapt"
Modify	Scale/reshape attribute A	scamper_op: "modify"
Put to other use	Repurpose for goal G	scamper_op: "put_to_other_use"
Eliminate	Remove component X entirely	scamper_op: "eliminate"
Reverse	Invert relationship R	scamper_op: "reverse"

2. Bisociation (Koestler, 1964)

Connects two unrelated "matrices of thought" — domain-specific conceptual frames. The AnalogicalReasoner (20.2) supplies cross-domain analogies that seed these matrices.

3. Morphological Analysis (Zwicky, 1969)

Decomposes the problem into attributes, lists values for each, then enumerates combinations:

Attribute     │ Value 1        │ Value 2        │ Value 3
──────────────┼────────────────┼────────────────┼───────────
Interface     │ CLI            │ GUI            │ Voice
Storage       │ Graph DB       │ Vector DB      │ Hybrid
Processing    │ Batch          │ Stream         │ Event-driven

Combinations: 3 × 3 × 3 = 27 candidate ideas from just 3 attributes.

---

Integration Map

Phase 13.3 CuriosityModule                Phase 20.2 AnalogicalReasoner
    │ curiosity_score > threshold              │ cross-domain analogies
    │ → high-curiosity concepts                │ → bisociation matrices
    ▼                                          ▼
┌──────────────────────────────────────────────────┐
│              DivergentGenerator (22.1)            │
│  seed_pool ◄── curiosity concepts                │
│  matrices  ◄── analogical mappings               │
│  selection_pressure ◄── emotional valence (21.1) │
└──────────────────────┬───────────────────────────┘
                       │ evolved idea population
                       ▼
              Phase 22.2 ConvergentEvaluator
                       │ filtered + ranked ideas
                       ▼
              Phase 22.3 CreativeBlender
                       │ blended solutions
                       ▼
              Phase 22.5 CreativityOrchestrator

CuriosityModule (13.3) → Seeds the generator with high-curiosity concepts. When curiosity_score exceeds a threshold, the concept is added to _seed_pool, ensuring the generator focuses on genuinely interesting territory.

AnalogicalReasoner (20.2) → Supplies structural analogies across domains. These become the "matrices of thought" for bisociation — e.g., an analogy between "ant colony optimization" and "distributed consensus" creates a bisociation opportunity.

EmotionModel (21.1) → Emotional valence modulates selection pressure. Ideas that evoke surprise or curiosity get a tournament bonus, while ideas that trigger negative valence (frustration, confusion) get a penalty — mimicking how human creativity is guided by emotional intuition.

---

Metrics & Observability

Five Prometheus metrics provide full visibility:

# Grafana panel: Idea Generation Rate by Strategy
- panel: timeseries
  title: "Ideas Generated / min (by strategy)"
  query: rate(asi_divergent_ideas_generated_total[5m])
  legend: "{{strategy}}"

# Grafana panel: Novelty Score Distribution  
- panel: histogram
  title: "Novelty Score Distribution"
  query: asi_divergent_novelty_score_bucket

# Alert: Population stagnation
- alert: DivergentPopulationStagnant
  expr: asi_divergent_population_size < 10
  for: 5m
  summary: "Divergent population below minimum — generation may be stuck"

---

Open Questions

1. Embedding model: Should score_novelty() use the same embedding model as WorldModel (13.1), or a creativity-specific model trained on divergent outputs?
2. Population persistence: Should the evolved population persist across CognitiveCycle ticks, or reset each cycle?
3. Multi-objective novelty: Should we add a usefulness dimension alongside novelty, or leave that to ConvergentEvaluator (22.2)?

---

Spec: #514 · Phase 22 Planning: (coming soon)