separability-index-correlated-irreducibility

IN premise — summaries/2026/08/24/engels-2024-not-all-features-linear-s4-s-parse-autoencoders-find-multi-d-imensional-features.md

Created 2026-08-25T02:58:01+00:00

A higher separability index S(f) (minimum mutual information over all rotations) indicates a feature is more separable and thus more likely irreducible.

Summary

If a feature stays strongly distinguishable no matter how you rotate or reframe the data, it is doing essential, independent work that cannot be broken down into simpler pieces. This gives the system a concrete, quantitative way to tell which features are fundamental and must be preserved versus which are redundant or could be decomposed away.