security-surfaces-expand-with-capability-scaling

IN derived (depth 4)

Created 2026-06-21T10:25:10+00:00 · Reviewed 2026-06-21T14:41:08+00:00

The early evidence of dual-scaling tensions between capabilities and risks (as illustrated by GPT-2's memorization and misuse concerns) compounds the challenge posed by three independent security surfaces — training data poisoning, prompt injection, and architectural vulnerabilities — since architectural vulnerabilities in particular appear fundamental rather than solvable by scale alone, suggesting that LLM security may be a persistently difficult problem rather than one that straightforward engineering progress will resolve.

Summary

LLM security is unlikely to get easier just because models improve — the core vulnerabilities in how these systems process instructions are structural, not bugs that better engineering or bigger models will fix. For the system, this means security work is a permanent, multi-front cost across training, architecture, and inference, rather than a problem with a clear end date.

Justifications

SL — Each security surface grows with capability, preventing convergence on static defense solutions

Antecedents (all must be IN):

IN llm-security-requires-defense-across-three-independent-surfaces — LLM security threats operate across three independent attack surfaces requiring distinct defenses: training data poisoning (deliberate grooming of web content), architectural prompt sensitivity (40%+ accuracy shifts from formatting, instruction-input confusion), and inference-time injection — and the architectural vulnerabilities are fundamental, not solvable by engineering or scale.
IN gpt2-foreshadowed-capability-risk-dual-scaling — GPT-2's staged release over misuse concerns, combined with measurements showing 1-7% exact duplicate training data in its outputs, illustrated early tensions between scaling language models and managing associated risks such as memorization and potential misuse.

Dependents

These beliefs depend on this one:

OUT accessibility-constraints-provide-inadvertent-security-buffer — The agentic paradigm's accessibility constraints — technical complexity of the optimization stack, legal restrictions on model use, and safety classification barriers — function as an inadvertent security buffer, limiting the exposure of expanding attack surfaces by restricting who can deploy capable agent systems at scale.
IN dual-use-properties-make-security-scaling-superlinear — LLM security challenges tend to scale with capability growth because dual-use properties like memorization — which serves as both a knowledge source and an attack vector — mean that mechanisms driving capability improvements can simultaneously deepen vulnerabilities. Evidence from training data memorization patterns and the expansion of multiple independent security surfaces (poisoning, prompt injection, architectural vulnerabilities) suggests that security surfaces don't merely grow alongside capabilities but may become structurally harder to defend, though this pattern is characterized at limited scales rather than confirmed as a universal structural law.
IN security-challenge-compounds-across-all-maturity-dimensions — LLM security is uniquely difficult because challenges compound across every dimension of the field's maturity: attack surfaces expand with capability scaling, defense-in-depth strategies are simultaneously necessitated and bounded by theoretical gaps, and the empirical nature of the field means security expertise — like all operational expertise — is irreducibly experiential rather than formally transferable.