❓ Q&A 22.1 — DivergentGenerator: divergent thinking strategies, novelty scoring & genetic algorithm

[web3guru888]

❓ Q&A 22.1 — DivergentGenerator

Answers to common questions about the DivergentGenerator (Phase 22.1, spec: #514).

---

Q1: Why five divergent strategies instead of just one?

A: Different strategies explore different regions of the creative search space:

Strategy	Exploration Type	Best For
RANDOM_COMBINATION	Combinatorial	Connecting existing concepts in new ways
SCAMPER	Systematic transformation	Incremental innovation on existing ideas
BISOCIATION	Cross-domain transfer	Breakthrough/radical innovation
CONSTRAINT_RELAXATION	Boundary exploration	Finding hidden assumptions
MORPHOLOGICAL_ANALYSIS	Exhaustive enumeration	Complete coverage of attribute space

Using all five maximises ideational flexibility (Guilford's divergent thinking dimension). A single strategy would produce many ideas, but they'd cluster in one region. The strategy rotation in generate() ensures the population spans multiple creative dimensions from the start.

---

Q2: How does the novelty scoring formula work?

A: Novelty is a weighted combination of two distance metrics:

novelty = 0.6 · cosine_distance(idea, centroid) + 0.4 · min_nearest_neighbor

• Centroid distance (α=0.6): How far is this idea from the "average" idea in the population? High centroid distance means the idea is globally different.
• Nearest-neighbor distance (β=0.4): How far is this idea from its closest neighbor? High NN distance means the idea occupies a sparse region — it's not a minor variation of something that already exists.

The two terms complement each other: centroid distance rewards global divergence, while NN distance rewards local uniqueness. An idea can be far from the centroid but close to one neighbor (a cluster outlier) — the NN term penalises this.

Edge case: Empty population returns 1.0 (maximum novelty), because the first idea has no population to be similar to.

---

Q3: What are the genetic algorithm parameter trade-offs?

A: The DivergentConfig defaults represent a balanced configuration:

Parameter	Default	↑ Higher	↓ Lower
population_size	50	More diversity but slower	Faster but less diversity
max_generations	10	Better convergence but slower	Faster but may miss optima
crossover_rate	0.7	More recombination (exploitation)	More cloning (exploration)
mutation_rate	0.15	More exploration, risk of chaos	More exploitation, risk of stagnation
tournament_size	3	Stronger selection pressure (elitist)	Weaker selection (more random)
novelty_threshold	0.3	Stricter filtering, smaller population	Permissive, larger population

Key insight: In creative search, we want more exploration than exploitation compared to traditional GA. That's why crossover_rate (0.7) is high but tournament_size (3) is moderate — we recombine freely but don't aggressively select, preserving population diversity.

The mutation_rate of 0.15 is deliberately low because each mutation is a semantic perturbation (synonym substitution, concept injection), which can radically change meaning even at low rates.

---

Q4: How does the DivergentGenerator integrate with the CognitiveCycle?

A: The DivergentGenerator plugs into the CognitiveCycle as a creativity sub-step:

# In CognitiveCycle._run_step()
async def _creative_step(self) -> None:
    """Phase 22 creative problem-solving sub-step."""
    # 1. Get high-curiosity concepts from CuriosityModule (13.3)
    seeds = await self._curiosity.get_high_curiosity_concepts(threshold=0.7)
    
    # 2. Feed seeds into DivergentGenerator
    for seed in seeds:
        self._divergent.add_to_seed_pool(seed.concept)
    
    # 3. Generate diverse ideas for the current problem
    ideas = await self._divergent.evolve_population(prompt=self._current_goal)
    
    # 4. Pass to ConvergentEvaluator (22.2) for filtering
    evaluated = await self._convergent.evaluate(ideas)
    
    # 5. Top ideas become candidate plans for GoalDecomposer (10.2)
    for idea in evaluated[:5]:
        await self._goal_decomposer.propose_subgoal(idea.content)

The creative step runs after the reasoning step (Phase 20) and before the planning step (Phase 10), allowing creative solutions to feed into the planning pipeline.

---

Q5: What is bisociation and how does it differ from simple analogy?

A: Bisociation (Arthur Koestler, The Act of Creation, 1964) is fundamentally different from analogy:

• Analogy: Maps structure from a known source domain to a target domain. The connection is recognised and systematic. Example: "An atom is like a solar system."
• Bisociation: Connects two previously unrelated "matrices of thought" (habitual frames of reference) in a way that creates a new meaning neither domain contained alone. Example: Gutenberg connecting wine-press mechanics with coin-stamping to invent the printing press.

In the DivergentGenerator:

• AnalogicalReasoner (20.2) discovers analogies — systematic mappings between domains
• _bisociate() uses these as raw material but goes further — it randomly pairs concepts from unrelated matrices, looking for surprising bridges that weren't part of any systematic mapping

The key metric difference: analogical ideas tend to have moderate novelty (they're structured), while bisociative ideas have high novelty variance (many are nonsensical, but the rare hits are breakthrough-level).

---

Q6: How does the seed pool work with CuriosityModule?

A: The _seed_pool is a bounded deque (maxlen=100) of concept strings that serve as raw material for all generation strategies:

CuriosityModule (13.3)
    │ get_high_curiosity_concepts(threshold=0.7)
    │ → ["quantum error correction", "neuromorphic computing", ...]
    ▼
┌──────────────────────────────────────────┐
│ _seed_pool (deque, maxlen=100)           │
│ ┌────────────────────────────────────┐   │
│ │ "quantum error correction"         │   │
│ │ "neuromorphic computing"           │   │
│ │ "swarm intelligence"               │   │
│ │ "biomimetic materials"             │   │
│ │ ... (FIFO, oldest evicted first)   │   │
│ └────────────────────────────────────┘   │
└──────────────────┬───────────────────────┘
                   │
    ┌──────────────┼──────────────────┐
    ▼              ▼                  ▼
RANDOM_COMBINE  BISOCIATE    MORPHOLOGICAL
(pick 2-3)    (matrix seeds)  (attribute values)

The deque's FIFO eviction ensures the seed pool stays fresh — as the system's curiosity shifts, old seeds are naturally replaced by new ones. This creates a temporal bias toward recent curiosity, which is desirable: the system generates creative ideas about what it's currently curious about, not what interested it 1000 cycles ago.

---

Q7: How do we prevent the genetic algorithm from converging to a single idea cluster?

A: Several mechanisms maintain population diversity:

1. Multi-strategy seeding: The initial population comes from 5 different strategies, ensuring structural diversity from generation 0.

2. Novelty-based fitness: Unlike traditional GAs that optimise a fixed objective, we select for novelty — being different from the population. This creates a natural diversity pressure: as one cluster grows, its members' novelty drops, and outliers become fitter.

3. Nearest-neighbor penalty: The β·min_NN term in the novelty formula specifically penalises ideas that are close to any existing idea, even if they're far from the centroid.

4. Moderate tournament size (k=3): With only 3 candidates per tournament, weaker (but unique) ideas still have a ~1/3 chance of winning. Larger k would be more elitist.

5. Mutation as exploration: The 15% mutation rate introduces random perturbations that can jump to unexplored regions. Even after crossover converges two subtrees, mutation can break out.

6. Threshold-based filtering, not ranking: We discard ideas below novelty 0.3, but we don't rank-select within the surviving population. This preserves moderate-novelty ideas that might be stepping stones.

If the population still converges (all ideas above threshold but very similar), the CreativityOrchestrator (22.5) can detect this via the asi_divergent_population_size gauge and trigger a diversity injection — generating a fresh batch with CONSTRAINT_RELAXATION to break the local optimum.

---

Spec: #514 · Show & Tell: #519