irreducible-reducible-empirical-thresholds

IN premise — summaries/2026/08/24/engels-2024-not-all-features-linear-sR-references-chunk-1.md

Created 2026-08-25T02:58:02+00:00

Empirically, irreducible features show S(f) ≈ 2.7 bits and M_ε ≈ 0.18, while reducible features show S(f) ≈ 0.37 bits and M_ε ≈ 0.64.

Summary

This is an observed rule of thumb that lets the system tell apart two kinds of features by their numbers: irreducible (fundamental) ones carry roughly seven times more information and show much less overlap with other features, while reducible (derivable) ones are low-information and heavily redundant. In practice, it means the system can classify a new feature as fundamental or redundant just by measuring its entropy and redundancy scores against these two clusters.