saey-attention-only-rewritable-without-decomposition

Summary

Attention-only transformer layers have a clean algebraic structure that lets you flatten out internal state references without ever needing to identify what "features" the network is computing. The moment a ReLU feed-forward block is added, that clean structure breaks, meaning the network's internal representations can no longer be eliminated by rewriting and must instead be treated as opaque objects. This draws a sharp boundary: if you want a network you can surgically edit or reason about without feature-level assumptions, attention-only designs are the last stop before the analysis gets intractable.