superposition-introduces-positive-dot-products
IN premise — summaries/2026/08/24/elhage-2022-toy-models-superposition-chunk-11.md
Created 2026-08-25T02:58:00+00:00
Superposition introduces positive dot products between unrelated features, making naive cosine-similarity probes (e.g., to 'deceptive behavior' embeddings) unreliable for detecting feature presence.
Summary
When too many concepts are packed into the same activation space, they inevitably overlap, so measuring similarity against one concept's direction picks up signal from many unrelated concepts at the same time. In practice, this means a simple "does this activation point toward deceptive behavior?" check gives inflated, misleading scores, and the system needs more careful feature-separation methods to avoid false positives.