compression-quality-makes-capability-vulnerability-informationally-inseparable
IN derived (depth 3)
Created 2026-06-21T11:44:45+00:00 · Reviewed 2026-06-21T14:41:08+00:00
Language modeling's information-theoretic foundation — where model quality directly measures data compression capability (Chinchilla) — implies that memorization's dual-use nature is not a fixable flaw but an information-theoretic inevitability: better compression necessarily means more faithful reproduction of training data, making capability and vulnerability fundamentally the same phenomenon viewed from different angles.
Justifications
SL — Information theory proves capability (compression) and vulnerability (memorization) are the same mechanism
Antecedents (all must be IN):
- IN chinchilla-grounds-scaling-in-information-theory — Chinchilla research established that language model quality directly measures data compression capability (compressing ImageNet to 43% vs PNG's 58%), grounding compute-optimal scaling laws in information-theoretic foundations rather than purely empirical curve-fitting.
- IN memorization-is-dual-use-capability-and-vulnerability — Training data memorization exhibits dual-use characteristics: the same retention mechanism that contributes to model knowledge also creates an attack surface for deliberate data poisoning, as memorization rates serve as a quantitative proxy for poisoning vulnerability. GPT-2's early demonstration of both measurable memorization (1-7% exact duplicates) and capability-related safety concerns suggests this tension scales with model capability, though the evidence characterizes the pattern at one scale rather than confirming it as a universal structural property.
Dependents
These beliefs depend on this one:
- IN capability-vulnerability-inseparability-makes-security-unpatchable — The training data security surface is not merely permanently permeable after weight release but fundamentally unpatchable: since language model quality directly measures compression capability and memorization is informationally inseparable from that compression, removing memorized vulnerabilities necessarily degrades the model's core competence — the vulnerability IS the capability.
- IN paradigm-resilience-propagates-vulnerability-via-information-theoretic-identity — The pretrain-finetune paradigm's propagation of vulnerability across the training pipeline is grounded in information-theoretic identity: capability and vulnerability are not merely co-located but informationally inseparable (compression quality equals memorization capability), meaning the paradigm's three-dimensional resilience (production validation, architectural survival, RLHF embedding) cannot be preserved while excising the vulnerability it carries.