prompt-vulnerability-is-dual-architectural-limitation
IN derived (depth 2)
Created 2026-06-21T10:06:23+00:00 · Reviewed 2026-06-21T14:41:08+00:00
Both prompt sensitivity (40%+ accuracy shifts from formatting) and prompt injection (inability to distinguish instructions from input) appear to be intrinsic architectural properties of instruction-following LLMs that resist improvement through scaling alone. Both phenomena may share a common factor in the model's lack of formal prompt structure parsing, though the antecedents establish each independently rather than proving a unified root cause.
Summary
Both the fragility of LLM outputs to prompt formatting and the vulnerability to prompt injection trace back to the same architectural gap: the model has no built-in mechanism to formally separate and prioritize different layers of instructions, treating them all as one undifferentiated stream. This means neither problem will simply go away with a bigger model or more training, so deployed systems must be designed around these permanent constraints rather than expecting them to be patched out.
Justifications
SL — Two independently established depth-1 vulnerabilities are actually manifestations of the same architectural limitation: LLMs treat all input as undifferentiated tokens
Antecedents (all must be IN):
- IN prompt-sensitivity-is-fundamental-not-solvable-by-scale — LLM performance sensitivity to prompt formatting (40%+ accuracy shifts from reordering, 76-point swings from format changes) persists across model sizes, few-shot counts, and instruction tuning — it is an intrinsic property, not a fixable deficiency.
- IN prompt-injection-is-architectural-vulnerability-not-bug — Prompt injection is an architectural vulnerability inherent to instruction-following LLMs — arising from the fundamental inability to distinguish developer-defined instructions from user-supplied inputs — rather than a fixable implementation bug, making it the primary and potentially irreducible security concern for deployed applications.
Dependents
These beliefs depend on this one:
- IN llm-security-requires-defense-across-three-independent-surfaces — LLM security threats operate across three independent attack surfaces requiring distinct defenses: training data poisoning (deliberate grooming of web content), architectural prompt sensitivity (40%+ accuracy shifts from formatting, instruction-input confusion), and inference-time injection — and the architectural vulnerabilities are fundamental, not solvable by engineering or scale.
- IN mathematical-completeness-fails-to-guarantee-practical-reliability — RLHF and prompting both illustrate cases where formal or mathematical specification proves insufficient for practical reliability: RLHF has a fully specified mathematical pipeline yet naive implementations fail without dozens of engineering details (motivating simpler alternatives like DPO/IPO/KTO), while prompting exhibits irreducible sensitivity and architectural injection vulnerabilities rooted in the model's inability to formally parse prompt structure. These two examples suggest that in at least some core LLM techniques, mathematical completeness or formal specification does not guarantee practical reliability.
- IN prompt-fragility-compounds-with-context-expansion — The dual architectural limitation of prompt control — sensitivity (40%+ accuracy shifts from formatting) and injection (inability to distinguish instructions from input) — compounds with the 10,000x context window expansion enabled by efficiency breakthroughs: larger contexts create proportionally larger attack surfaces for injection attempts, make sensitivity effects harder to diagnose across expanded input, and increase the probability that adversarial content co-occurs with legitimate instructions in the same context window.