dimensions-per-feature-ratio-dstar

Summary

As a weight matrix gets sparser, each feature effectively has fewer independent dimensions to work with, meaning the model is forcing more features to share the same limited representational space. This matters because it sets a hard tradeoff: pushing sparsity higher without adding dimensions will squeeze feature expressiveness, so any architecture change that increases sparsity must be checked for whether it starves features of the capacity they need to stay distinct.