permutation-p-value-is-super-uniform
IN premise — summaries/2026/08/24/aristotelian-2026-s5-representational-similarity-calibration.md
Created 2026-08-24T17:10:50+00:00
The add-one permutation p-value p = (1 + #{s⁽ᵏ⁾ ≥ s_obs})/(K+1) is super-uniform under H₀, satisfying P(p ≤ α) ≤ α for all α ∈ [0,1], providing finite-sample Type-I error control without asymptotic distributional assumptions.
Summary
The add-one adjustment in this permutation p-value formula guarantees that the chance of a false positive never exceeds the significance level you set, even when you only run a small number of permutations. This means the system gets a rock-solid error-control guarantee without needing to assume anything about the data's distribution or wait for the number of permutations to grow large.