rome-causal-tracing-3sigma-corruption
IN premise — summaries/2026/08/24/meng-2022-rome-sR-references-chunk-1.md
Created 2026-08-25T02:58:16+00:00
Causal Tracing requires corrupting subject token embeddings with at least 3σ_t Gaussian noise; noise at σ_t produces effects too small to discern the indirect causal effect.
Summary
To meaningfully measure how much a model's output depends on a particular subject token's indirect influence, you have to disrupt that token's internal representation with noise at least three times the typical scale of its values; anything less washes out into background variation and you simply cannot tell whether an indirect causal effect exists. This sets a hard lower bound on how aggressively you must intervene before the measurement is valid, meaning under-powered causal tracing runs will silently report "no indirect effect" even when one is present.