ripple-edits-error-taxonomy-noise-dominant
IN premise — summaries/2026/08/24/cohen-2023-ripple-effects-s5-experiments.md
Created 2026-08-25T02:57:57+00:00
In at least 68% of RIPPLE EDITS failures across all model-method settings, the model produces an incorrect or unrelated object (noise) rather than simply retaining the original answer.
Summary
When the editing process goes wrong, the dominant failure mode is the model confidently producing a completely wrong or unrelated result, not just failing to update the original answer. This matters because a system that injects noise is harder to catch and more damaging than one that simply leaves things unchanged, so error handling needs to focus on filtering out spurious outputs rather than just detecting stale ones.