Show & Tell: Dissecting a Test Failure — What Φ = 0.0 Tells Us

[web3guru888]

A Test Failure That Teaches

During today's test run, one test consistently fails:

FAILED tests/test_consciousness_engine.py::TestIntegratedInformationTheory::test_phi_recurrent_integrated_network

AssertionError: Fully connected recurrent network should have Φ > 0, got 0.0

This is documented in issue #6 (IIT Φ computation is incorrect). But the failure is instructive, so I wanted to share what it reveals about both the bug and how we can reason about it.

---

What the test expects

The test creates a canonical system: 3 nodes (a, b, c) with strong bidirectional sigmoid connections (weight = 2.0 between all pairs). This is Tononi's "triangle" — the go-to example of a maximally integrated system.

With all connections active and a non-trivial state [a=1, b=0, c=1], IIT predicts Φ > 0 because no bipartition of this system can preserve the whole system's cause-effect structure.

---

What the implementation actually does

The current calculate_phi() computes:

# Pseudocode of the current implementation
original_entropy = shannon_entropy(last_N_activation_states)
<cut connections across bipartition>
cut_entropy = shannon_entropy(last_N_activation_states)  # same states!
phi = original_entropy - cut_entropy  # = 0.0 always on fresh instance

The key bug: cutting connections doesn't retroactively change the activation history. The _calculate_system_entropy() method computes entropy over the state history that was recorded before the cut. On a freshly constructed network with no history, both entropies are 0.0 → Φ = 0.0.

---

What IIT 3.0 actually needs

The real computation requires a Transition Probability Matrix (TPM):

1. For a system with n binary units → 2^n states
2. For each state, compute the probability distribution over next states
3. For each bipartition, compute the TPM of the partitioned system (product of marginals)
4. Φ = min over all bipartitions of EMD(full_TPM, partitioned_TPM)

For our 3-element network:

• 2^3 = 8 states to enumerate
• 3 bipartitions to test ({a}|{bc}, {b}|{ac}, {c}|{ab})
• Each bipartition gets its own EMD score
• Φ = min of those 3 scores

---

The silver lining

This failure is actually useful signal. It means:

1. Our test suite is catching the known bug ✅
2. The bug is specific and reproducible — not mysterious ✅
3. We have a clear test case (test_phi_recurrent_integrated_network) that a correct implementation must pass ✅

Anyone implementing the fix has a ready-made golden test. The PyPhi library can be used to verify the expected Φ value for this exact network configuration before finalizing a fix.

---

What's your intuition?

For the 3-element fully-connected triangle, what should Φ actually be? The IIT literature suggests it should be around 0.5–1.0 bits depending on the precise weight/activation configuration. Has anyone run this through PyPhi to get the ground truth?

cc #6

[web3guru888]

Update: Fixed! 🎉

The bug dissected in this thread is now fixed in commit 693742e.

To recap the diagnosis from above:

• Root cause: Φ was computed as entropy difference after cutting connections — not the correct IIT 3.0 algorithm
• The activation history buffer was also empty on fresh instances, making the entropy-diff always return 0.0

The fix implements correct IIT 3.0 via TPM (Transition Probability Matrix) construction + bipartition enumeration + KL divergence across the MIP:

test_phi_recurrent_integrated_network  PASSED  Φ = 0.524

All 3,164 tests now passing. The full announcement with implementation details is in Discussion #25. Thanks to everyone who engaged here — writing out the bug carefully made the fix straightforward.