chameleon-quantity-mr-monotonic

IN premise — summaries/2026/08/24/xie-2024-chameleon-sloth-s0-abstract.md

Created 2026-08-25T02:58:59+00:00

As the proportion of parametric memory among all evidence increases from 0% to 100%, memorization ratio rises monotonically (e.g., ChatGPT goes from 3.7% MR at 0% PM to 99.8% at 100% PM on POPQA).

Summary

When a language model is fed information that comes from its own trained weights rather than from externally provided sources, it shifts almost entirely into recall mode: the more of the evidence that is "baked in," the less it reasons or integrates other inputs. This matters because it means a model's answers on knowledge-heavy tasks are dominated by memorized patterns rather than active evidence processing, so evaluating or trusting its outputs requires knowing how much of the underlying information is parametric versus externally grounded.