relu-hidden-layer-model-equation

IN premise — summaries/2026/08/24/elhage-2022-toy-models-superposition-chunk-9.md

Created 2026-08-25T02:58:01+00:00

The privileged-basis ReLU hidden layer model is defined as h = ReLU(Wx), x' = ReLU(W^Th + b), where the ReLU activation breaks rotational symmetry and makes W directly interpretable.

Summary

This model uses a ReLU activation in a way that locks the weight matrix into a specific, unique orientation, so you can actually read the weights and understand what the network has learned rather than facing the usual ambiguity where any rotation of the weights produces the same behavior. In practice, this means the learned features are directly visible in the matrix W, which makes the network inspectable, debuggable, and comparable across runs in a way that standard dense layers do not allow.