attention-entropy-mechanism-correlational

IN premise — summaries/2026/08/24/convergence-without-understanding-2026-s3-results.md

Created 2026-08-24T17:10:52+00:00

Attention entropy H(a) = −Σ aᵢ log aᵢ over input positions shows a negative Pearson correlation (r = −0.41 to −0.48) with problem difficulty across 6 models, proposed as the mechanism by which hard problems produce diffuse, model-agnostic attention that homogenizes cross-model representations (correlational, not causal).

Summary

When models face harder problems, their attention spreads thin across many input positions rather than locking onto a few key ones, and this pattern holds consistently across six different architectures. The implication is that difficulty may push all models toward a similar, undifferentiated way of processing input, making their internal representations converge even though the models are built differently — though this link is a correlation, not yet proven causation.