mistral-alpha-copy-layers-15-17
IN premise — summaries/2026/08/24/engels-2024-not-all-features-linear-s8-we-then-evaluate-interventions-using-this.md
Created 2026-08-25T02:58:02+00:00
In Mistral 7B, the weekday index α is copied to the final output token at layers 15–17, explaining a drop in intervention effectiveness at those layers.
Summary
By layers 15 through 17, the Mistral 7B model has already locked in its answer by copying the chosen index into the output slot, so any attempt to steer or override the response at those layers falls flat. In practice, this means you need to intervene earlier in the network if you want to actually change what the model produces.