zhou-2023-memorization-ratio-formula
IN premise — summaries/2026/08/24/zhou-2023-context-faithful-prompting-s4-experiments.md
Created 2026-08-25T02:59:10+00:00
The Memorization Ratio (MR) is defined as MR = po / (po + ps), where po is the frequency of predicting the original (memorized) answer and ps is the frequency of predicting the substituted (counterfactual) answer; a fully faithful model has MR = 0.
Summary
This defines a score that measures how often a model clings to an answer it has seen before rather than adapting when the question is changed to a counterfactual version. The closer the score gets to zero, the more the model is genuinely understanding the task rather than just reciting memorized patterns, which is the hallmark of a faithful system.