logit-difference-attribution-formula
IN premise — summaries/2026/08/24/templeton-2024-scaling-monosemanticity-chunk-7.md
Created 2026-08-25T02:58:39+00:00
Logit-difference attribution ranks features by the quantity ∂(logit_A − logit_B) / ∂activation_i, measuring each feature's estimated causal contribution to the model's choice between two candidate next tokens.
Summary
This defines a specific scoring rule for explaining why the model picked one next token over another: each internal feature gets a number showing how much nudging it would shift the model's preference between the two candidates. In practice, it lets you decompose a single token-choice into a ranked list of which internal signals drove the decision, making the model's otherwise opaque selection process inspectable and attributable.