aggregation-calibration-order-matters

IN premise — summaries/2026/08/24/aristotelian-2026-sR-references.md

Created 2026-08-24T17:10:51+00:00

Calibrating per-element similarity scores and then aggregating (e.g., taking max) is statistically invalid; one must calibrate the aggregated statistic T(S) against the permutation null {T(S^(k))} to preserve super-uniformity (Proposition C.4)

Summary

If you calibrate each individual similarity score first and then combine them (say, by taking the max), the resulting test statistic will have distorted p-values, making your significance claims unreliable. The correct procedure is to run the aggregation first and then calibrate the combined statistic against what random permutations would produce, ensuring your "reject the null" decisions are actually valid.