sae-encoder-nonnegative-steering-negative
IN premise — summaries/2026/08/24/templeton-2024-scaling-monosemanticity-chunk-10.md
Created 2026-08-25T02:58:36+00:00
The SAE encoder output is always nonnegative, but the steering procedure can clamp a feature's activity to negative values, which is equivalent to subtracting a multiple of the feature vector from the residual stream.
Summary
SAE features are designed to only activate in the positive direction, so their natural range is zero to some positive value. Steering breaks that one-sided constraint by allowing a feature to go negative, which effectively means you are subtracting along that feature's direction rather than adding to it. This matters because it means steering can suppress or invert a feature's contribution to the residual stream, not merely amplify it, giving the operator a full two-sided control that the SAE's own architecture never intended.