fv-deltalogprob-formula

IN premise — summaries/2026/08/24/todd-2023-function-vectors-s2-function-vector-causal-effects-cannot-be-recovered-from-the--chunk-3.md

Created 2026-08-25T02:58:41+00:00

The vocabulary-space causal effect is computed as Δlogprob(w_i) = log P(w_i | h_ℓ + v_t) − log P(w_i), which ranks tokens by how much the FV promotes them beyond the baseline distribution.

Summary

This is the rule the system uses to score how strongly a feature pushes individual vocabulary items: it compares the chance a token appears when the feature is active against the chance it appears by default, and ranks tokens by the size of that difference. It turns "this feature influences output" into a concrete, per-word measurement that can be compared across tokens to see which ones the feature actually promotes.