Other

759 beliefs (476 IN, 283 OUT)

• IN 40pct-probe-vs-generation-gap
  Knowledge-probe accuracy exceeds generation accuracy by approximately 40%, indicating factual knowledge is stored in parameters but not reliably retrieved during decoding (Li et al., 2023c).
• IN accessibility-barriers-are-inherently-experiential
  The NLP field's empirically-driven engineering progression likely contributes to accessibility barriers that are substantially experiential — formal documentation alone may not fully bridge the gap because much of the critical knowledge emerged through iterative practice and accumulated engineering mastery rather than being derived from theory, creating tacit-knowledge barriers that published research may struggle to capture.
• IN accessibility-constraints-are-tacit-knowledge-barriers
  The agentic paradigm's accessibility constraints are likely reinforced by the broader pattern of engineering maturity outpacing formal understanding — deployment depends on mastering implementation details within comprehensive optimization stacks that resist theoretical specification, contributing to tacit-knowledge barriers that published research alone may not fully bridge.
• OUT accessibility-constraints-provide-inadvertent-security-buffer
  The agentic paradigm's accessibility constraints — technical complexity of the optimization stack, legal restrictions on model use, and safety classification barriers — function as an inadvertent security buffer, limiting the exposure of expanding attack surfaces by restricting who can deploy capable agent systems at scale.
• IN activation-function-affects-polysemanticity
  Kshitij Sachan at Redwood Research independently replicated the Elhage et al. 2022 toy model experiments and found that the choice of activation function (e.g., ReLU vs. alternatives) materially affects the degree of polysemanticity and shifts the phase boundary.
• IN adoption-acceleration-and-deployment-divergence-compound-security-exposure
  The capability-deployment gap (widened by the adoption flywheel pushing capabilities into production at unprecedented rates) and the training-deployment optimization divergence (training prioritizes data volume while deployment requires efficiency, quantization, and memory management) create compound security exposure: adoption acceleration means security-unvalidated capabilities reach production faster, while training-deployment divergence means security properties established during training don't reliably transfer to deployed systems.
• IN adoption-acceleration-compounds-craft-knowledge-bottleneck
  The adoption flywheel's acceleration of the capability-deployment gap compounds the inherently experiential nature of the field's accessibility barriers: capability advances outpace experiential knowledge transfer, and this knowledge deficit cannot be bridged through documentation or formal training alone.
• IN adoption-and-persistence-create-compound-risk-multiplier
  The adoption flywheel and persistent memory create a compound risk multiplier: adoption acceleration pushes continuous agents into production at a rate that outpaces security expertise development, while persistent memory adds cross-session attack surfaces that compound with each deployment cycle.
• IN adoption-flywheel-accelerated-paradigm-convergence-on-agentic-ai
  The compounding adoption flywheel — where Transformer generality expands the addressable market and alignment enables mass adoption, which funds further capability development — appears to have accelerated what might otherwise have been a more gradual NLP evolution toward the agentic paradigm. While the antecedents establish that alignment ignited mass adoption and that the Transformer's architectural flexibility enabled cross-domain generalization, the specific causal links between investment flows and capability timelines remain underspecified. The compression from chatbot to autonomous agent occurred rapidly (roughly 2022–2025), but the degree to which this flywheel — as opposed to other factors — accounts for the speed of that convergence is not fully established by the available evidence.
• IN adoption-flywheel-amplifies-capability-deployment-gap
  The compounding adoption flywheel — where alignment enables adoption, which funds capability, which expands the addressable market — actively widens the gap between capability diffusion and responsible deployment capacity, because each cycle accelerates capability faster than governance frameworks and operational expertise can scale.
• OUT adoption-flywheel-converges-safely-without-regulatory-intervention
  The adoption flywheel's market dynamics — where alignment enables adoption and adoption funds further capability and safety research — converge toward safe deployment without requiring external regulatory intervention.
• OUT agentic-autonomy-demands-alignment-where-rlhf-is-weakest
  The agentic paradigm, enabled by massive context window expansion, increases the importance of robust alignment — yet RLHF, the dominant alignment technique, exhibits a completeness paradox (mathematically specified but practically fragile) that has driven the field toward both simpler alternatives and more complex engineering workarounds, leaving open questions about whether current alignment approaches are well-matched to the demands of autonomous, long-context agents.
• OUT agentic-deployment-is-safe-at-scale
  Frontier models' convergence on multimodal agentic capabilities is safely deployable at scale — validated by Claude Code's 5.5x revenue growth demonstrating market acceptance — provided that prompt injection does not represent an irreducible architectural vulnerability in instruction-following systems.
• IN agentic-paradigm-requires-context-alignment-and-efficiency-convergence
  The agentic application paradigm appears to depend on at least two converging developments: massive context window expansion — enabled by efficiency breakthroughs addressing quadratic attention costs — which created a prerequisite substrate for stateful autonomous operation, and the concurrent diversification of alignment approaches (RLHF, DPO family, Constitutional AI), a coincidence that may prove relevant if different alignment methods offer distinct advantages for the varied deployment contexts (code, GUI, visual design) that agentic systems operate across.
• IN agentic-paradigm-technically-enabled-but-accessibility-constrained
  The agentic paradigm is enabled in part by context window expansion (itself dependent on efficiency breakthroughs addressing quadratic attention), but its practical deployment is constrained by compounding accessibility barriers — comprehensive optimization stacks, restrictive licensing, and safety classification — contributing to a gap between demonstrated capability and deployable reality.
• OUT agentic-paradigm-viable-despite-training-deployment-divergence
  The agentic paradigm remains viable despite the fundamental divergence between training optimization (data volume priority, exponential cost scaling) and deployment optimization (inference efficiency, multi-layer optimization stack), because the efficiency breakthroughs that enabled context expansion also bridge both requirements — but only if the inference-layer security architecture holds.
• IN ai-detection-tools-66-68-percent-accuracy-baseline
  AI-detection tools achieve only 66–68% accuracy, no better than human baselines, and AI-generated text scores 100% on plagiarism detectors.
• IN air-canada-tribunal-liability-feb-2024
  In February 2024, a tribunal held Air Canada liable for its chatbot's hallucinated bereavement policy, rejecting the 'separate legal entity' defense.
• OUT alignment-approaches-are-mature-enough-for-reliable-deployment
  The diversification of alignment into three independent mathematical paradigms (RLHF, DPO family, Constitutional AI) with proven production results (ChatGPT, Claude) means alignment is a mature engineering discipline with reliable deployment pathways.
• IN alignment-bootstrap-creates-circular-safety-assurance
  The alignment system's circular dependency is deeper than its bootstrap origin: alignment is a product of the craft methodology it compensates for, AND the training pipeline masks the fundamental capacity inversion between pretraining and alignment — meaning the craft methodology that cannot formally verify safety also masks the very capacity asymmetry that makes alignment insufficient, creating a circular safety assurance where the evaluation method is blind to the failure mode it should detect.
• IN alignment-compensation-addresses-only-fraction-of-total-safety-deficit
  Alignment diversification compensates for the formal verification deficit at the alignment layer, but the structural and widening safety deficit operates at the capability layer — meaning the alignment fix addresses only a shrinking fraction of the total safety challenge, which continues to grow independently with each capability advance.
• IN alignment-diversification-compensates-for-craft-formal-verification-deficit
  Alignment diversification into multiple independent paradigms serves as a practical substitute for formal safety verification: since the craft discipline fundamentally cannot provide formal guarantees for any single alignment approach (safety assurance is inherently informal), and RLHF's irreducible theoretical complexity (non-Markovian optimal policies, online/offline divergence) rules out complete formal specification even for the best-understood approach, having multiple independent alignment paths provides probabilistic coverage that no single formally unverifiable approach can offer alone.
• IN alignment-diversification-inevitable-yet-insufficient
  Alignment diversification was theoretically inevitable (RLHF's irreducible complexity demanded alternatives) yet practically insufficient — even three independent alignment paradigms compensate only for the formal verification deficit at the alignment layer, while the structural safety deficit operates across training, deployment, and security dimensions that no alignment method can reach.
• OUT alignment-diversification-provides-genuine-safety-redundancy
  The three independent alignment paradigms (RLHF, DPO/KTO, Constitutional AI) provide genuine safety redundancy by compensating for the craft discipline's formal verification deficit through methodological diversity — each paradigm's blind spots are covered by the others' independent theoretical foundations.
• IN alignment-diversification-was-theoretically-inevitable
  Alignment diversification beyond RLHF was theoretically inevitable rather than merely pragmatically convenient: RLHF's irreducible theoretical complexity (non-Markovian optimal policies, online/offline formulation divergence) combined with the broader pattern of mathematical completeness failing to guarantee practical reliability left no viable path to reliable alignment through a single paradigm.
• IN alignment-diversified-into-three-independent-paradigms
  LLM alignment diversified from a single RLHF pipeline into three independent paradigms — full mathematical RLHF, direct preference optimization (DPO/IPO/KTO), and Constitutional AI — each eliminating different sources of complexity while preserving alignment quality.
• OUT alignment-diversity-ensures-safe-capability-scaling
  The diversification of alignment into three independent paradigms (RLHF, DPO/KTO, Constitutional AI), combined with the proven capability-adoption flywheel, should provide adequate safety headroom as frontier models scale — multiple independent alignment approaches mean no single failure mode can compromise the entire safety stack.
• OUT alignment-diversity-resilient-against-preference-signal-failures
  The diversification of alignment into three independent paradigms (RLHF, DPO/KTO, Constitutional AI) provides resilience against paradigm-specific failures — if one approach has systematic blind spots, the independent alternatives can compensate, and the orthogonality of reasoning and alignment means failures in one dimension do not cascade to the other.
• IN alignment-evaluation-doubly-circular-and-vulnerable
  The alignment system is doubly compromised: alignment is a product of the craft methodology it compensates for (circular bootstrap), and its primary evaluation mechanism (the RLHF reward model) inherits training data vulnerabilities from the same pretrain-finetune paradigm whose safety deficit it is supposed to measure — the system that evaluates alignment quality is itself vulnerable to the risks that motivate alignment in the first place.
• IN alignment-ignited-capability-adoption-feedback-loop
  ChatGPT's demonstration that alignment enables mass adoption, combined with frontier models' subsequent convergence on multimodal agentic capabilities, suggests a plausible reinforcing dynamic: alignment helped unlock adoption (ChatGPT), adoption likely contributed to funding capability expansion (multimodal, agentic features), and expanded capabilities may require more sophisticated alignment — positioning alignment as a potential catalyst for an ongoing cycle rather than a one-time gate.
• IN alignment-is-bootstrap-product-of-craft-methodology-it-compensates
  The LLM field's alignment mechanisms (RLHF, DPO, Constitutional AI) are themselves products of the craft methodology whose limitations create the safety deficit they are meant to address — alignment diversification compensates for the craft discipline's lack of formal verification, yet each alignment paradigm was developed, validated, and deployed using the same empirical craft methods, creating a bootstrap dependency where the solution inherits the epistemology of the problem.
• IN all-training-objectives-are-compression-variants
  The three main Transformer training task types — masked language modeling (BERT-style bidirectional), autoregressive/causal (GPT-style left-to-right), and PrefixLM (hybrid bidirectional prefix with causal generation) — are all variants of self-supervised compression, and Chinchilla's information-theoretic grounding (model quality directly measures compression capability) reveals they differ in compression strategy rather than in kind.
• IN alpac-report-1966-cut-mt-funding
  The ALPAC report (1966) led to dramatic funding cuts for machine translation research after the Georgetown experiment's overpromising
• IN alpac-report-landmark-mt-critique
  The ALPAC report (1996) is a frequently cited historical touchstone documenting the limitations and failures of early rule-based machine translation systems.
• IN alpaca-30b-50pct-closebook-consistency
  Alpaca-30B is consistent in only approximately 50% of close-book QA cases (Li et al., 2023d).
• OUT angular-encoding-not-radius
  Off-distribution grid interventions (sweeping r ∈ [0,2] and θ ∈ [0,2π]) show the model encodes weekday/month index in the angle θ of the circular representation, not the radius r.
• OUT anthropic-lifecycle-management-supports-responsible-scaling
  Anthropic's comprehensive model lifecycle management — structured deprecation, scheduled retirement, weight preservation for ethically motivated reasons — combined with Claude's rapid expansion into agentic platforms demonstrates that aggressive capability scaling and responsible stewardship can coexist.
• OUT anthropic-safety-approach-balances-capability-and-responsibility
  Anthropic's safety approach — Constitutional AI alignment, tiered safety classification (Level 3 for Opus 4), and refusing DoD compromises on surveillance/weapons ethics — represents a coherent responsible deployment model.
• IN aristotelian-2026-icml-venue-and-arxiv
  'Revisiting the PRH: An Aristotelian View' by Gröger, Wen, and Brbić (EPFL) was published at ICML 2026 (43rd ICML, Seoul, PMLR 306), with preprint arXiv:2602.14486v2 [cs.LG] dated 25 Jun 2026.
• IN asp-inertia-rule-strong-negation-syntax
  The Answer Set Programming inertia rule is expressed as: r(X, T+1) ← r(X, T), not ~r(X, T+1), using strong negation to express 'the fluent is not explicitly made false.'
• IN attention-efficiency-enables-the-security-risk-it-cannot-address
  The attention efficiency breakthroughs that were existential prerequisites for the agentic paradigm also enabled the persistent memory capabilities that amplify compounding security challenges — meaning the same technical achievements that opened the agentic frontier simultaneously contributed to some of its most difficult security problems, though the antecedents do not establish whether architectural mechanisms could decouple the enabling efficiency from the resulting risk.
• IN attention-efficiency-is-existential-prerequisite-for-agentic-paradigm
  The agentic paradigm depends existentially on attention efficiency breakthroughs: attention's validated universality makes it irreplaceable as the computational primitive, its quadratic cost makes efficiency research existential for deployment, and these efficiency gains are what enabled the context expansion that made agentic applications possible.
• OUT attention-is-all-you-need-2017-google
  The 'Attention Is All You Need' paper (Vaswani et al., 2017, NeurIPS) introduced the Transformer architecture and was authored by 8 Google researchers
• IN attention-universality-grounds-entire-efficiency-research-agenda
  Attention's validated universality across domains (grounded in its structural computational richness — asymmetry, position-dependence, learned scaling) makes the efficiency research it demands existential for the entire field: the comprehensive efficiency stack is not merely optimizing one implementation choice but resolving the fundamental cost constraint of the field's only proven universal computation primitive.
• IN attention-universality-makes-efficiency-existential
  Attention's validated status as a universal computation primitive — the sole mechanism underlying all frontier language, protein, chess, and RL models — transforms its O(n²) complexity from a performance concern into an existential constraint: the technique that everything depends on is the one most expensive to scale.
• IN attention-universality-validates-craft-epistemic-methodology
  Attention's validated universality across domains (language, protein folding, chess, reinforcement learning) — grounded in its structural computational richness (asymmetry, position-dependence, learned scaling) — was discovered empirically rather than predicted by theory, providing one of the strongest validations that the craft discipline's empirical methodology can discover genuinely deep structural properties even without theoretical foundations to guide the search.
• IN autoepistemic-logic-non-monotonicity-mechanism
  Autoepistemic logic is non-monotonic because it contains formulae expressing 'not known'; learning a new fact removes the corresponding 'not-known' formula, so addition of knowledge causes removal of conclusions.
• OUT automated-prompt-compilation-could-close-inference-theory-gap
  The convergence of automated prompt engineering (LLMs generating human-quality prompts) with declarative LM pipeline compilation (DSPy optimizing multi-step LM programs) could close the systematic theory gap in prompt engineering — transforming inference-time control from craft-dependent tuning into a verifiable, compilable engineering discipline.
• IN backfire-effect-repilation-wood-2019
  Wood et al.'s replication study with 10,100 participants across 52 issues found no evidence for the backfire effect originally claimed by Nyhan & Reifler (2010), leading to the conclusion that the effect is rare or nonexistent.
• IN bacon-novum-organum-1620-idols
  Francis Bacon's 'Novum Organum' (1620) identified 'idols of the mind' as a source of confirmation bias, representing one of the earliest written descriptions of the phenomenon.
• IN base-models-lower-accuracy-428-vs-508-percent
  Base models show lower mean accuracy (42.8%) than instruction-tuned models (50.8%) on the 800-problem evaluation set, reflecting difficulty extracting structured answers without instruction-tuning.
• OUT base-models-show-stronger-inversion-than-instruction-tuned
  The difficulty inversion is stronger in non-instruction-tuned base models (ΔCKA = +0.117, monotonic 0.962 → 0.827 across bins) than in instruction-tuned models (ΔCKA = +0.067), ruling out RLHF/RLAIF alignment training as the source of the phenomenon.
• OUT bayes-conjugate-prior-same-family
  A conjugate prior ensures the posterior is in the same parametric family as the prior (e.g., Beta prior × Bernoulli likelihood → Beta posterior).
• OUT bayes-posterior-proportional-to-likelihood-times-prior
  Because the marginal likelihood P(E) is the same across all competing hypotheses, the posterior is proportional to the product of prior and likelihood: P(H|E) ∝ P(E|H)·P(H).
• OUT bayes-theorem-formula
  Bayes' Theorem states P(H|E) = P(E|H)·P(H) / P(E), where P(E) is the marginal likelihood (evidence) that serves as a normalizing constant.
• IN beck-cognitive-therapy-early-1960s
  Aaron Beck developed cognitive therapy in the early 1960s, with its core mechanism being teaching patients to evaluate evidence impartially rather than selectively reinforcing negative outlooks.
• IN bengio-2000-neural-probabilistic-language-model-distributed-representation
  Bengio et al. (2000) introduced neural probabilistic language models that learned dense distributed word representations, replacing the sparse high-dimensional vectors of earlier vector space models and directly motivating modern word embeddings
• OUT bengio-2003-first-modern-neural-language-model
  Bengio et al. (2003) introduced the neural probabilistic language model, and Mikolov et al. (2010) extended it to RNN-based language models, forming the direct lineage preceding the Transformer era.
• IN bengio-2003-mlp-outperformed-ngram
  Bengio et al. (2003) showed that a multi-layer perceptron language model outperformed word n-gram models, providing early evidence for neural language modeling
• IN bengio-2003-neural-language-model
  Bengio et al. (2003) demonstrated that neural networks (multi-layer perceptrons) outperform n-gram language models, a key precursor to modern LLMs
• IN bert-base-12l-768h-110m
  BERT_BASE has 12 layers, 768 hidden size, and 110M parameters; BERT_LARGE has 24 layers, 1024 hidden size, and 340M parameters
• IN bert-deployment-validated-pretrain-finetune-at-scale
  Google's deployment of BERT to Search — processing every English query by October 2019 and expanding to 70+ languages by December 2019 — provided the first large-scale production validation that the pretrain-finetune paradigm works reliably beyond benchmarks.
• OUT bert-encoder-only-bidirectional
  BERT uses only the encoder portion of the transformer with bidirectional (all-to-all) self-attention and no causal masking, making it unable to generate text
• OUT bert-encoder-only-gpt-decoder-only
  BERT is an encoder-only model (masked prediction) while GPT is a decoder-only model (autoregressive next-token prediction).
• IN beyer-1999-nn-distance-ratio-converges-to-1
  Beyer, Goldstein, Ramakrishnan & Shaft (1999) proved that in sufficiently high dimensions with i.i.d. features, the ratio of farthest-to-nearest-neighbor distances converges to 1, rendering nearest-neighbor search meaningless.
• IN bingo-basket-polarization-oral-report
  In the bingo-basket polarization experiment, only the group that verbally stated their probability after each draw showed polarization; the group that reported only at the end did not.
• IN both-control-layers-structurally-degrade-under-agentic-scaling
  The complete dual-layer LLM control stack degrades structurally under agentic scaling: training-time alignment diversification addresses only a fraction of the total safety deficit, while inference-time control self-undermines as context windows expand — leaving no compensating layer as either degrades.
• OUT bpe-tokenization-process
  Byte-pair encoding (BPE) tokenization starts with all unique characters as unigrams, iteratively merges the most frequent adjacent pairs, and repeats until the vocabulary reaches a prescribed size
• IN bpe-ulm-vocabularization-plus-segmentation-algorithms
  BPE and ULM each include both a vocabularization algorithm (building the vocabulary) AND a segmentation algorithm (splitting text into tokens); an alternative is rule-based greedy left-to-right segmentation over a pre-trained vocabulary.
• IN bret-taylor-serves-as-openai-board-chair
  Bret Taylor serves as board chair of OpenAI; former board members have included Ilya Sutskever (co-founder) and Elon Musk (early backer).
• OUT brown-2020-gpt3-few-shot-learners
  Brown et al. (2020) 'Language Models are Few-Shot Learners' first demonstrated in-context learning at scale using GPT-3 with 0-shot, 1-shot, 5-shot, and 32-shot framings.
• OUT c-lexer-hack-typedef
  The lexer hack in C arises because the token class of an identifier (typedef name vs variable name) cannot be determined without semantic information, requiring information flow from the semantic analyzer back to the lexer.
• IN c-lexer-hack-typedef-semantic-feedback
  The C lexer hack requires semantic feedback from the symbol table to distinguish typedef names from variable names, since they are lexically identical.
• IN c2023-4-category-failure-taxonomy
  Cohen et al. (2023) define a 4-category failure taxonomy for post-edit responses: correct edit, no change, abstention, and noise
• IN cOUNTERFACT-dataset-purpose
  The COUNTERFACT dataset contains deliberately factually incorrect requested rewrites (e.g., changing a linebacker's position to 'goaltender') to test whether an editor correctly resists invalid edits.
• OUT cad-decoding-formula
  CAD's decoding formula is P_CAD(token) ∝ P₁(token) / [P₀(token)]^α, where P₁ is the context-conditioned distribution and P₀ is the model's prior-knowledge distribution.
• IN calibration-noise-distribution-invariance
  Permutation-based null calibration correctly collapses scores to zero under Gaussian, Student-t, Laplace, and Gaussian mixture noise without requiring any distributional assumptions about the data-generating process
• IN capability-diffusion-outpaces-responsible-deployment-capacity
  The open-weight ecosystem exhibits a structural mismatch between capability diffusion and deployment readiness: weight availability tends to outpace governance frameworks (accidental leaks catalyze adoption, definitions of "open" remain contested), while frontier capability convergence is independently constrained by safety classification and licensing restrictions — the result is that raw model access can spread widely even as the conditions for responsible deployment lag behind.
• OUT capability-risk-dual-scaling-proved-systemic
  GPT-2's early demonstration of capability-coupled risks (1-7% memorization, staged release over misuse concerns) proved prescient rather than incidental — the same dual scaling pattern compounded into today's multi-layered safety challenge spanning capability classification (Level 3), government suspension directives (Fable 5/Mythos 5), architectural vulnerabilities (prompt injection), and calibration failures (over-refusal), confirming capability-risk co-scaling as a systemic property, not an early-stage artifact.
• IN capability-vulnerability-inseparability-makes-security-unpatchable
  The training data security surface is not merely permanently permeable after weight release but fundamentally unpatchable: since language model quality directly measures compression capability and memorization is informationally inseparable from that compression, removing memorized vulnerabilities necessarily degrades the model's core competence — the vulnerability IS the capability.
• IN capacity-inversion-hidden-beneath-maximally-intractable-agents
  Continuous agents — the most intractable deployment challenge in the field — operate on a foundation whose most dangerous structural flaw is permanently hidden: the capacity inversion between pretraining and alignment (where over-parameterization helps pretraining but constrains alignment) is masked by craft validation, meaning the highest-capability systems are built atop an invisible structural weakness in their safety layer.
• IN capacity-inversion-permanently-invisible-at-scale
  The fundamental capacity inversion between pretraining and alignment is permanently invisible at scale: craft-based empirical validation masks the inversion because production success substitutes for formal diagnosis, while the expertise scalability paradox ensures that practitioners who might develop the theoretical sophistication to recognize it can never reach sufficient density in the exponentially growing field.
• IN cca-vs-sammon-opposite-distance-focus
  Curvilinear Component Analysis (CCA) focuses on preserving small output-space distances (its stress function relates to a sum of right Bregman divergences), while Sammon's mapping focuses on preserving small input-space distances
• IN centering-theory-joshi-weinstein-1981
  Centering theory, introduced by Joshi and Weinstein (1981), is a foundational discourse-structure model for controlling inference in discourse.
• OUT cgd-approximated-single-middle-layer
  In Shen et al. (2023), cGD (calibrated Gradient Descent) is approximated by performing gradient descent on a single random middle layer of LLaMA rather than fine-tuning the full model.
• IN cgd-learning-rates-tested
  The cGD experiments in Shen et al. test four learning rates: 1e-4, 5e-4, 1e-3, and 5e-3.
• OUT cgd-simulated-on-single-random-middle-layer
  In Shen et al. (2023), the continual gradient descent (cGD) baseline is simulated by optimizing on a single randomly chosen middle layer of LLaMA rather than the full model, as a deliberate simplification.
• OUT cgd-single-middle-layer-optimization
  The cGD simulation in Shen et al. (2023) optimizes one random middle layer of LLaMA as a partial fine-tune, not full fine-tuning.
• OUT chameleon-memorization-ratio-formula
  Memorization Ratio is defined as MR = f_m / (f_m + f_c), where f_m is the frequency of the memory-answer selection and f_c is the frequency of the counter-answer selection.
• IN chatgpt-demonstrated-alignment-enables-adoption
  ChatGPT's November 2022 launch — applying RLHF to GPT-3.5 — illustrated that alignment techniques like RLHF, applied on top of capable base models, can play a significant role in transforming raw capability into widely adopted products.
• IN chatgpt-fragmentation-inconsistency-46-9pct
  When a single piece of evidence is split into fragments, ChatGPT changes its preference in approximately 46.9% of cases (53.1% consistency rate).
• OUT chatgpt-launched-2022-11-30
  ChatGPT was launched on 2022-11-30, triggering the mainstream AI boom.
• OUT chatgpt-public-release-november-2022
  ChatGPT was publicly released in November 2022, after which 'hallucination' became the dominant industry term for LLM factual errors.
• OUT chinchilla-balance-prescribes-optimal-resource-allocation
  Chinchilla's prescription to scale parameters and data in equal proportion provides the optimal training resource allocation strategy.
• IN chinchilla-information-theory-predicts-parameter-redundancy
  Chinchilla's information-theoretic grounding — establishing that language model quality directly measures data compression capability — theoretically predicts the parameter redundancy observed empirically (95% performance at 60% of parameters, cross-layer sharing, weight tying): if language modeling IS compression and models are over-parameterized relative to the compression target, redundancy is an expected theoretical consequence, not merely an empirical surprise.
• OUT chinchilla-information-theory-provides-principled-scaling-guidance
  Chinchilla's information-theoretic foundation — establishing that language modeling is compression and predicting parameter redundancy — provides principled guidance for model scaling decisions rather than purely empirical groping, offering a theoretical compass for the craft discipline's otherwise empirically-driven methodology.
• OUT chinchilla-scaling-balance-size-and-data
  Chinchilla scaling research suggests optimal training balances model parameter count and training data size, rather than simply scaling parameters alone.
• IN circular-encoding-angle-not-radius
  In the off-distribution grid sweep (r∈[0,2], θ∈[0,2π)), the target value α is encoded in the angle θ of the circular subspace, not the radius r; sweeping r at fixed θ does not change the decoded answer.
• OUT cka-difficulty-inversion-gap-specific-values
  The difficulty inversion CKA gap is approximately 0.067 for linear CKA, 0.091 for RBF-kernel CKA, +0.061 for MNN (k=10), and +0.054 for SVCCA.
• OUT cka-null-baseline-scales-as-o-d-over-n
  Under the null hypothesis of independence, the CKA similarity baseline scales as O(d/n), meaning the null interaction energy E[‖Ĉ‖²_F] = d_x·d_y/(n−1) is non-zero in high-dimensional regimes even when two representations are truly independent.
• OUT claude-2-1-context-window-200k-tokens
  Claude 2.1 (November 2023) doubled the context window to 200,000 tokens (approximately 500 pages of text).
• OUT claude-code-launch-feb-2025-ga-may-2025
  Claude Code launched as a preview in February 2025 and reached general availability in May 2025.
• OUT claude-code-source-leaked-source-map-march-2026
  Claude Code CLI source code was leaked in March 2026 via an exposed source-map file.
• OUT claude-constitutional-ai-alignment
  Anthropic uses Constitutional AI for alignment, employing a written constitution of principles rather than extensive human feedback. The 2026 constitution is 23,000 words, up from 2,700 in 2023.
• OUT claude-constitutional-ai-early-udhr
  Early versions of Anthropic's Constitutional AI constitution drew from the UN Universal Declaration of Human Rights.
• OUT claude-named-after-claude-shannon
  Anthropic's Claude model family is named after Claude Shannon, the founder of information theory
• OUT claude-opus-4-agents-are-production-capable
  Claude Opus 4-class models demonstrate production-grade agentic capability, as shown by 16 Opus 4.6 agents writing a C compiler in Rust.
• OUT claude-opus-4-level-3-safety
  Opus 4 was the first Claude model classified Level 3 ('significantly higher risk') on Anthropic's four-point safety scale
• OUT claude-three-tier-naming-haiku-sonnet-opus
  Claude models are released in three standard size tiers: Haiku (smallest/cheapest), Sonnet (mid-tier), and Opus (largest/most capable).
• IN claudetbot-ifixit-1m-requests-jul2024
  Anthropic's 'ClaudeBot' web crawler hit the iFixit website approximately 1 million times per day in July 2024 while ignoring anti-AI-scraping robots.txt policies.
• IN cnn-early-layers-encode-lowlevel-features-typically-frozen
  In CNN fine-tuning, early (input-adjacent) layers encode low-level features such as edges and textures and are typically frozen, while later layers encode task-specific high-level features and are fine-tuned.
• IN coarticulation-makes-speech-recognition-hard
  Coarticulation — where sounds of successive letters blend together in natural speech — is a major factor making speech recognition difficult
• IN coherence-boosting-mechanism
  Coherence boosting (Malkin et al., 2021) demotes a short-context model's distribution from the full-context distribution to emphasize long-range coherence, a related but distinct contrastive pair compared to CAD's no-context vs. full-context approach.
• IN compiler-theory-illuminates-tokenizer-model-separation
  The formal language hierarchy separating lexer-level processing (regular languages, finite-state machines) from parser-level processing (context-free languages, pushdown automata) has a structural analog in LLM architecture: subword tokenization (BPE/WordPiece) operates at the finite-state statistical level while attention provides the unbounded context-sensitivity that finite-state processing cannot, illuminating why tokenization remains a separate preprocessing stage rather than being absorbed into the model's core computation.
• OUT complete-llm-lifecycle-is-standardized-but-fragile
  The full LLM lifecycle — from standardized training pipeline through diversified alignment paradigms to deployment — has matured into a reproducible engineering discipline with known mathematical foundations, proven production results, and multiple independent alignment paths.
• IN complete-llm-paradigm-requires-efficiency-to-be-practical
  The full NLP paradigm shift culminating in decoder-only dominance only became a practical deployment paradigm because the comprehensive efficiency stack (Flash Attention, speculative decoding, MoE, PagedAttention) addressed attention's quadratic scaling constraint — without efficiency breakthroughs, decoder-only dominance would be a theoretical victory unusable at frontier scale.
• IN complete-memory-hierarchy-creates-complete-vulnerability-hierarchy
  The memory hierarchy — bridging horizontal capacity expansion (10,000x context windows) and vertical persistence (cross-session state consolidation) — creates a corresponding expanded vulnerability surface: the context expansion paradox (simultaneously enabling and undermining the agentic paradigm) extends across temporal levels from intra-turn through inter-episode to cross-session, with persistent memory amplifying both the enabling and undermining dimensions beyond any single session's scope.
• OUT compound-risk-manageable-through-craft-self-correction
  The compound risk from adoption acceleration pushing continuous agents into production could be managed through the craft discipline's empirical self-correction — deployment feedback naturally concentrating practitioner attention on the most dangerous failure modes first, as the same experiential learning that characterizes the field's knowledge accumulation would surface and patch vulnerabilities through production observation.
• IN compressed-sensing-recovery-bound
  In classical compressed sensing, an n-dimensional k-sparse vector can be recovered from an m-dimensional projection if and only if m = Ω(k log(n/k)).
• IN compression-is-the-fundamental-scaling-variable
  Since all transformer training objectives — masked, autoregressive, and prefix language modeling — are variants of data compression, and empirical evidence consistently shows data volume outweighs parameter count for model performance, the fundamental scaling variable is the quality and quantity of information being compressed, not the capacity of the compressor.
• IN compression-quality-makes-capability-vulnerability-informationally-inseparable
  Language modeling's information-theoretic foundation — where model quality directly measures data compression capability (Chinchilla) — implies that memorization's dual-use nature is not a fixable flaw but an information-theoretic inevitability: better compression necessarily means more faithful reproduction of training data, making capability and vulnerability fundamentally the same phenomenon viewed from different angles.
• IN confidence-91-3pct-single-source-96-3pct-multi-source
  LLMs show >95% confidence in 91.3% of single-source counter-answer examples and 96.3% of multi-source memory-aligned answer examples.
• IN confidence-extraction-1000-samples-per-scenario
  Confidence extraction for Llama2-7B samples 1,000 examples per scenario and computes normalized log probability over the three candidate answer tokens.
• IN confidence-measurement-three-token-normalization
  Confidence is measured by normalizing log probabilities over three candidate answer tokens (memory answer, counter-answer, uncertain) and reporting the distribution of the first generated token's probability.
• OUT configuration-explosion-manageable-within-craft-discipline
  The combinatorially explosive configuration space at the innovation frontier is manageable within the craft discipline's empirical methods — practitioners can navigate it through iterative experimentation and accumulated heuristics without requiring formal theoretical guidance.
• IN configuration-space-explosion-at-innovation-frontier
  The innovation frontier now operates in a combinatorially explosive configuration space: the shift from settled macro-architecture to actively contested micro-architecture coincides with orthogonal reasoning and alignment dimensions that multiply independently across training and inference layers.
• IN confirmation-bias-cognitive-vs-statistical-taxonomy
  Confirmation bias is classified as a cognitive bias, distinct from statistical biases (estimator, sampling, selection, survivorship, omitted-variable) and institutional biases (publication, funding, media).
• IN confirmation-bias-not-self-fulfilling-prophecy
  Confirmation bias (biased information processing) is distinct from the self-fulfilling prophecy (expectations changing behavior to produce the expected outcome).
• IN confirmation-bias-three-channels-search-interpretation-recall
  Confirmation bias operates through three distinct mechanistic channels: biased search (seeking confirming information), biased interpretation (setting higher evidentiary standards for contradictory information), and biased recall (selectively remembering expectancy-congruent evidence).
• IN conflict-dataset-sizes
  Benchmark dataset sizes per the survey include KC (9,803 CM examples), KRE (11,684 CM), WikiContradiction (2,210 IC), Pan et al. 2023a (52,189 IC), and PARAREL (328 IM).
• OUT conjugate-prior-posterior-same-family
  A conjugate prior keeps the posterior distribution in the same parametric family as the prior, simplifying Bayesian computation.
• IN conspiracy-theory-confirmation-bias-mechanism
  Confirmation bias is listed as one of several core psychological mechanisms underlying conspiracy-theory formation, alongside cognitive dissonance, communal reinforcement, attitude polarization, denialism, paranoia, and psychological projection.
• OUT constitutional-ai-anthropic-alignment
  Constitutional AI (developed by Anthropic) and RLHF (used by OpenAI) represent distinct alignment approaches for LLMs
• OUT constitutional-ai-scales-beyond-rlhf-complexity
  Constitutional AI provides a complete alignment path that bypasses RLHF's irreducible theoretical complexity (non-Markovian optimal policies, divergent online/offline formulations) by using written principles and AI-generated feedback instead of per-example human preferences — but only if training data memorization does not create attack surfaces that corrupt the base model's capacity to follow constitutional principles faithfully.
• IN context-and-retrieval-jointly-ceiling-limited
  Context engineering (as successor to prompt engineering) and retrieval augmentation (evolved from RAG to GraphRAG) are jointly constrained by the structurally unimprovable inference-time control layer — advances in either discipline cannot overcome the fundamental theory gap at the prompt processing level they both operate within.
• OUT context-as-infinite-dimension-complement
  The context window is the operational realization of the "infinite" or "unbounded" dimension in the superposition framework: while the d-dimensional residual stream is over-compressed to store N≫d features (necessitating covariance whitening for well-defined read/write operations), the context window provides an unbounded, interference-free channel where knowledge is injected without superposition, without the need for C⁻¹k* addressability, and without the rank-one constraint—making it the architectural complement to the finite, superposed parametric memory.
• OUT context-completes-tail-coordinate-space
  The context window is not "more memory" but the geometric completion of the addressable coordinate system: it specifically fills the tail *directions* absent from the rank-one parametric subspace, making the full semantic space (parametric + contextual) a closed coordinate system rather than two disjoint mechanisms.
• OUT context-engineering-enables-reliable-llm-deployment
  Context engineering — managing system instructions, retrieval context, token budgets, and provenance — provides a reliable methodology for deploying LLMs with consistent, controllable behavior.
• IN context-engineering-inherits-degrading-control-surface
  Context engineering inherits a degrading control surface: as the successor discipline to prompt engineering, it assumes responsibility for inference-time control just as that control becomes structurally unimprovable (resistant to formalization due to the systematic theory gap) while the memory surfaces it must manage multiply across temporal levels — it inherits a mandate it is structurally unable to fulfill.
• IN context-engineering-inherits-structural-limits-of-inference-control
  Context engineering, as the successor discipline to prompt engineering, inherits the structural limits of inference-time control: the same self-undermining dynamics — prompt fragility compounding with context expansion while the authority hierarchy serves as both defense and attack surface — apply to context engineering's broader scope of system instructions, retrieval context, token budgets, and provenance tracking, meaning the discipline's operational practices manage but cannot resolve the fundamental vulnerability.
• IN context-engineering-practices
  Context engineering emphasizes operational practices including token budgeting, provenance tags, versioning of context artifacts, observability logging, and context regression tests
• IN context-expansion-compounds-governance-gap
  The 10,000x context window expansion over seven years occurred alongside the governance gap created by weight availability: as model weights became increasingly accessible despite unresolved governance frameworks, released models also gained substantially larger context windows — suggesting that governance challenges may involve both growing numbers of accessible models and increasing capability per model, though the specific security implications of larger context windows require independent evidence beyond the expansion trend itself.
• IN context-expansion-enabled-agentic-application-paradigm
  The 10,000x context window expansion — made possible by efficiency breakthroughs addressing quadratic attention — created the prerequisite substrate for agentic applications, as demonstrated by Claude's evolution from a chatbot to CLI coding agent, GUI office automation, and visual design tool, all of which require processing large, stateful contexts.
• IN context-expansion-enabled-by-efficiency-breakthroughs
  The 10,000x context window expansion (1K to 10M tokens over seven years) was only achievable because efficiency research (Flash Attention, PagedAttention, linear attention variants) driven by quadratic attention costs made such scales computationally tractable.
• IN context-expansion-paradox-compounds-with-memory-persistence
  The agentic paradigm's foundational paradox — context expansion simultaneously enabling and undermining it, structurally permanently rather than transiently — is compounded by memory persistence amplifying security challenges: while the paradox was initially bounded to intra-session context windows, persistent cross-session memory extends the vulnerability surface beyond any single session's scope, making the paradox both more enabling (accumulating expertise across sessions) and more undermining (accumulating attack surface across sessions).
• IN context-expansion-paradox-structurally-permanent-not-transient
  The agentic paradigm's foundational paradox — context expansion simultaneously enables and undermines it — is structurally permanent rather than a transient growing pain: the inference-time control layer that context expansion degrades is resistant to formalization (systematic theory gap), meaning the vulnerability amplification cannot be engineered away even as the enablement deepens.
• IN context-expansion-simultaneously-enables-and-undermines-agentic-paradigm
  Context expansion is simultaneously the agentic paradigm's foundational enabler and its primary vulnerability amplifier: the same 10,000x context window growth that created the substrate for autonomous agent operation also created a proportionally larger attack surface for prompt injection and sensitivity failures — the paradigm's existence condition is its degradation mechanism.
• OUT context-extends-semantic-currency-to-infinite-dimension
  The context window extends the "universal semantic currency" (covariance geometry) into an effectively unbounded-dimensional space, making the full LLM read/write system a finite-dimensional-plus-infinite-dimensional geometric object rather than a purely finite one.
• OUT context-trend-as-geometric-convergence-in-practice
  The industry trajectory of expanding context windows (200K tokens), agentic tooling, and multi-hour autonomous runs is the operational convergence toward the theoretically-predicted "complete" semantic space: as the unbounded write channel grows, the parametric/contextual duality becomes a matter of degree rather than kind, empirically confirming the geometric prediction that the full d-dimensional space is the union of both channels.
• OUT context-window-ephemeral-edit
  The long-context window with near-perfect in-context key-value retrieval functions as an ephemeral, weight-free knowledge editing mechanism: injecting a fact into the 200K-token context is functionally equivalent to a rank-one edit that bypasses the parametric write subspace, providing a complementary write channel with zero persistence cost.
• OUT context-window-externalization-validation
  Long-context windows (200K tokens) with near-perfect in-context key-value retrieval provide operational validation of the externalization principle at practical scale, demonstrating that in-context storage is a reliable substitute for parametric long-tail knowledge when the model can attend to all relevant information simultaneously.
• OUT context-write-unbounded-complement
  The context window functions as the unbounded write channel that complements the rank-one parametric write: where parametric editing is confined to a single MLP's addressable subspace, the context channel injects arbitrary quantities of verified knowledge, making the two write channels capacity-complementary rather than redundant.
• IN contextual-knowledge-four-sources
  Contextual knowledge in LLMs comprises four sources: user prompts, multi-turn dialogue history, retrieved web documents (RAG), and tool/API outputs.
• IN continuity-very-early-late-weekday
  Prompts like 'very early on Monday' shift the layer-30 activation point along the circle toward Sunday, while 'very late on Monday' shifts toward Tuesday, demonstrating the model interpolates between discrete weekday categories.
• IN continuous-agents-are-apex-of-formally-ungrounded-engineering
  Continuous agents — agentic AI with persistent cross-session memory — represent the apex capability of an entirely empirically-driven engineering progression: the most autonomous and consequential LLM deployment mode (where errors persist and compound across sessions) was achieved at the terminus of a historical trajectory characterized throughout by engineering maturity outpacing theoretical understanding, meaning the capability with the highest stakes for safety has the least formal foundation for safety assurance.
• OUT continuous-agents-deployable-with-defense-in-depth
  Continuous agents with persistent memory are safely deployable at scale — persistent memory's paradigm-extending capability combined with five independent defense dimensions provides sufficient safeguards for long-horizon autonomous operation.
• IN continuous-agents-face-maximally-intractable-deployment
  Continuous agents face the most intractable deployment challenge in the field: as the apex of formally ungrounded engineering with maximal capability-risk coincidence, they reach production fastest via the adoption flywheel yet arrive precisely where deployment expertise is thinnest and security blind spots are widest.
• IN continuous-agents-maximize-capability-risk-coincidence
  Continuous agents represent a maximal capability-risk coincidence: as the apex of formally ungrounded engineering they are the most capable yet least formally understood capability, while their persistent memory maximally amplifies the already-compounding security challenges — the field's greatest achievement and greatest vulnerability are literally the same feature.
• IN contradiction-minimal-effect-confidence
  Despite producing wrong answers, LLMs do not lower their output confidence when exposed to contradictory context (Chen et al., 2022).
• OUT convergence-is-necessary-not-contingent
  Cross-model geometric convergence (SAE feature similarity, Park orthogonality) is a logical necessity of superposition in a shared residual stream rather than a contingent empirical coincidence: any system that encodes d concepts in an over-complete m > d basis within a common substrate MUST produce the same covariance structure
• IN core-llm-techniques-transferred-from-outside-nlp
  Two foundational LLM techniques — attention (evolved from NMT augmentation to standalone architecture) and RLHF (transferred from Atari/robotics via Christiano 2017) — originated outside NLP text generation and transferred successfully, each exploiting a domain-independent property (parallel computation for attention, easy-to-judge evaluation for RLHF). These two cases illustrate that cross-domain technique transfer has been one source of important LLM advances, though two examples alone do not establish its frequency or relative importance compared to NLP-native innovation.
• IN coreference-includes-bridging
  Coreference resolution includes bridging relationships (e.g., 'the front door' referring to a specific house's door), not just pronoun matching; anaphora resolution is the pronoun-specific subcase
• IN coreference-resolution-subsumes-anaphora-resolution
  Coreference resolution is the broader task encompassing anaphora resolution (pronoun→antecedent links) and bridging references (e.g., 'the front door' → John's house's front door).
• IN correctness-subspace-ablation-flips-3-to-7-percent-of-answers
  Projecting out the correctness subspace (spanned by probe-activating directions at the peak probe layer) flips model answers at rates of 3–7% across 8 small models (Qwen, SmolLM2, Gemma, LLaMA, Phi-3.5).
• OUT cot-threshold-validates-emergent-discontinuity
  Chain-of-thought prompting's empirically measured threshold of ~62B parameters is a specific documented instance of emergent abilities' discontinuous appearance at scale, validating that reasoning itself is an emergent property rather than a gradually improving one.
• IN counter-memory-length-ratio-1-2-threshold
  When counter-memory length ratio exceeds 1.2 relative to parametric memory, the LLM's answer distribution shifts significantly.
• OUT covariance-geometry-as-operational-semantic-space
  The covariance/whitening geometry (second-moment matrices) is the operational definition of semantic coordinate space in LLMs: it simultaneously parameterises feature interpretation (SAE decoder space, Park polytopes), similarity evaluation (cosine→Spearman pipeline), and knowledge modification (ROME rank-one updates), and this structure converges across model families.
• OUT covariance-geometry-unifies-analysis-and-editing
  The mathematically principled framework for both interpreting (SAE feature extraction, Park polytope analysis) and modifying (ROME rank-one edits) LLM representations is second-moment covariance geometry applied to the residual stream, since C = KKᵀ whitening defines the canonical coordinate system in which all three operations become linear algebra on the same substrate.
• OUT craft-architectural-convergence-guarantees-behavioral-stability
  The craft discipline's empirical methodology — producing convergent architectural choices across independent research groups (SwiGLU, RoPE, RMSNorm adopted by Llama, PaLM, and others via craft-by-assembly) combined with innovation transcending organizational boundaries — guarantees behaviorally stable architectures through empirical selection pressure.
• OUT craft-discipline-could-self-correct-via-innovation-boundary-crossing
  The craft discipline's fundamental epistemology — where innovation value correlates with boundary-crossing and the field's most valuable properties are discovered empirically — could self-correct its safety deficit through the same cross-boundary mechanism that drove its capability breakthroughs, importing safety formalization techniques from mature engineering disciplines.
• IN craft-discipline-nature-makes-safety-assurance-fundamentally-informal
  The LLM field's identity as a craft discipline — where both its most valuable properties and its accessibility barriers are empirical rather than formal — means safety assurance is fundamentally informal: security challenges that compound across all maturity dimensions cannot be formally verified in a field that discovers its own properties only through practice.
• IN craft-discipline-security-permanently-reactive
  The LLM field's identity as a craft discipline — where knowledge accumulates through deployment experience — combined with reactive security at an unpredictable frontier means that proactive security methodology is structurally impossible: craft knowledge requires accumulated experience, but the frontier's unpredictability ensures past experience does not transfer to novel capabilities, permanently confining security to reactive posture by structural necessity rather than institutional failure.
• OUT craft-discipline-self-correction-undermined-by-undetectable-threats
  The LLM field's craft discipline nature enables self-correction through empirical deployment feedback — practitioners discover both strengths and weaknesses through experience, creating a learning loop where the field improves by iterating on its own outputs.
• IN craft-epistemic-crisis-self-reinforcing-via-market-dynamics
  The unified craft-epistemic crisis appears structurally self-reinforcing: the craft methodology that generates both the safety deficit and the expertise crisis also shapes market dynamics that preferentially amplify innovations at the points where craft-based security awareness tends to be weakest — the field's epistemology simultaneously creates blind spots and economic forces that disproportionately reward activity in those blind spots.
• IN craft-knowledge-transfer-makes-innovation-value-institutionally-uncontainable
  Google's foundational contributions (Transformer, BERT) becoming the universal foundation for all frontier models — including direct competitors — is consistent with a structural property of craft disciplines: because craft knowledge in such fields transfers primarily through empirical practice and open publication rather than through institutional IP control, foundational innovations in a craft discipline tend to transcend their originating institution. The LLM field's craft-discipline character suggests this pattern of uncontainability is a likely structural feature rather than an accident of Google's specific choices.
• IN craft-methodology-unfalsifiably-self-consistent
  The LLM field's craft methodology is unfalsifiably self-consistent: its most rigorous quantitative results (scaling laws, information-theoretic constants) are themselves empirically discovered rather than theoretically derived, and NLP's AI-completeness ensures no formal methodology can replace empiricism — the craft validates itself by the only epistemology demonstrably available, creating a closed epistemic loop with no external reference point.
• IN craft-resilience-mechanism-is-scalability-bottleneck
  The pretrain-finetune paradigm succeeded precisely because it is craft-validated (empirical production survival, not formal proof), yet the expertise scalability paradox ensures this very craft nature prevents the methodology's tacit knowledge from being democratized at the rate the adoption flywheel demands — the mechanism of methodological resilience is simultaneously the mechanism of practitioner scarcity.
• IN craft-root-cause-unifies-expertise-crisis-and-resilience-bottleneck
  The dual expertise crisis (deployment AND security) and the craft resilience bottleneck share an identical root cause operating in opposite directions: the field's experiential knowledge-building mechanism is simultaneously the source of its most resilient contributions (pretrain-finetune survived because it was craft-validated at scale) AND the bottleneck preventing those contributions from scaling to meet demand (the same experiential requirement that validated pretrain-finetune makes expertise non-transferable).
• IN craft-validation-masks-capacity-inversion
  The craft discipline's empirical validation methodology actively masks the fundamental capacity inversion between pretraining and alignment: because the field validates by deployment outcomes rather than formal analysis, the training pipeline's standardized stages appear uniformly mature even though pretraining benefits from parameter redundancy while alignment faces strict capacity constraints requiring entirely different scaling strategies.
• IN cross-domain-import-export-cycle-is-craft-metaevidence
  NLP's complete cross-domain cycle (importing attention from machine translation, RLHF from robotics, then exporting transformer architecture to protein folding, chess, and reinforcement learning) constitutes empirical meta-evidence for the craft epistemic methodology — the cycle's most transformative transfers were discovered through deployment experience, not predicted by theory, validating the very methodology that produced them.
• OUT cross-modal-knn-alignment-degrades-with-gallery-scale
  Cross-modal mutual kNN alignment between vision and language models degrades as the evaluation gallery scales from 1,024 to 1,000,000 samples (WIT dataset), while within-modality mutual kNN alignment remains stable across the same range, as shown by Koepke et al. (2026).
• OUT cross-model-convergence-conditional-on-artifact-control
  The cross-model convergence of geometric structure (polytope geometry, feature universality, orthogonality) constitutes a genuine architectural property, but the orthogonality component specifically requires set-inclusion controls to distinguish genuine hierarchical semantics from combinatorial artifacts.
• OUT curse-of-dimensionality-coined-bellman-1957
  The term 'curse of dimensionality' was coined by Richard E. Bellman in 1957 in the context of dynamic programming, describing exponential growth of computational complexity as dimension count increases.
• IN dalle-text-to-image-sora-text-to-video-whisper-speech-to-text
  Within OpenAI's specialized models: DALL-E performs text-to-image generation, Sora performs text-to-video generation, and Whisper performs speech-to-text recognition (ASR), not speech synthesis.
• OUT data-scaling-paradigm-remains-safely-dominant
  The data-volume-first scaling strategy — independently validated by Chinchilla scaling laws and Llama's compression evidence — remains the dominant and safe approach to capability improvement, with massive web-scale data ingestion as the primary scaling lever.
• IN dax-med-relabeling-improved-hypothesis-testing
  Replacing 'fits/doesn't fit' labels with neutral 'DAX/MED' labels in Wason's task significantly improved participant hypothesis-testing performance, supporting the positive-test-strategy interpretation.
• IN debriefing-paradigm-partial-belief
  The debriefing paradigm (present evidence → measure → debrief → re-measure) demonstrates that some belief always survives full debriefing, even when participants report understanding and trusting the correction.
• OUT decision-transformer-validates-rl-subsumed-by-sequence-modeling
  Decision Transformer's successful recasting of reinforcement learning as autoregressive sequence modeling validates that the transformer's next-token prediction objective can subsume entire computational paradigms, reducing RL problems to the same sequence completion framework underlying all language modeling.
• OUT decoder-only-dominance-is-permanent-architectural-consensus
  Decoder-only architecture's dominance — driven by data scaling advantage and confirmed by the innovation frontier's shift from macro-architecture to micro-architectural configuration — represents a permanent architectural consensus, with no remaining structural incentive to revisit the encoder/decoder choice.
• IN decoder-only-scaling-requires-comprehensive-efficiency-stack
  The decoder-only architecture won the scaling race through data volume advantage, but its quadratic attention cost means this dominance is only practically sustainable through the comprehensive efficiency stack (Flash Attention, PagedAttention, speculative decoding, MoE).
• OUT deepseek-r1-671b-open-weight-jan-2025
  DeepSeek-R1 (January 2025) is a 671B-parameter open-weight reasoning model, comparable in capability to OpenAI o1 at lower cost.
• OUT deepseek-r1-671b-open-weight-reasoning
  DeepSeek-R1 (January 2025) is a 671B parameter open-weight reasoning model with performance comparable to OpenAI o1 at lower cost per token.
• IN defense-in-depth-required-and-bounded-by-theory-gap
  The five-dimensional defense requirement for LLM reliability is both necessitated and bounded by insufficient formal understanding — no single defense layer has theoretical guarantees (hence defense-in-depth is required), but the same theory gap means defense-in-depth itself lacks formal assurance of adequacy, creating an irreducible reliance on empirical validation.
• OUT democratized-inference-could-close-frontier-accessibility-gap
  Democratized inference — CPU-only execution eliminating GPU requirements and single-executable distribution eliminating installation complexity — could close the persistent frontier accessibility gap by removing the technical deployment barriers that persist despite capability convergence between proprietary and open-weight models.
• IN dense-regime-no-privileged-basis
  At S=0 (dense regime), there is no privileged feature basis; any set of m orthonormal directions is equally optimal, and geometric structure only emerges as sparsity increases.
• IN dependency-directed-backtracking-contradiction-resolution
  When a contradiction is detected in a TMS, dependency-directed backtracking identifies the responsible statements and updates dependency records, rather than performing blind backtracking.
• IN deployment-divergence-compounds-accessibility-gap
  The accessibility gap persists despite capability convergence in part because training and deployment require fundamentally divergent optimization strategies — organizations that achieve capability parity through data scaling and alignment still face a separate optimization challenge for production deployment, involving efficiency techniques to manage quadratic attention costs, while safety classification and licensing restrictions independently constrain which capabilities can be widely deployed.
• IN deployment-reliability-is-recursive-experiential-barrier
  LLM deployment reliability creates a recursive barrier: the irreducible operational expertise required for reliable deployment can only be acquired through deployment itself, and the experiential nature of all barriers in the field means no amount of pre-deployment study can substitute for this experience — creating a bootstrap problem where reliability knowledge presupposes the very deployment it's meant to enable.
• IN diffusion-iterative-denoising-vs-autoregressive-token-sampling
  Diffusion models (DALL-E, Stable Diffusion, Sora) generate via iterative denoising of a noise distribution, while LLMs generate via sequential autoregressive token sampling—fundamentally different generation mechanisms
• IN digital-nudging-two-forms
  Digital nudging has exactly two recognized forms: information nudging (disclaimers/warnings on source validity) and presentation nudging (exposing users to contrary viewpoints).
• IN dimensions-per-feature-ratio-dstar
  The dimensions-per-feature ratio D* = m / ||W||^2_F decreases as sparsity S increases, indicating more features are packed into the same number of dimensions.
• OUT directional-edits-as-microcosm-of-read-write-asymmetry
  ROME's directionality constraint (A→B requires a separate edit from B→A) is the micro-level instantiation of the macro read-broad/write-narrow asymmetry: the same over-complete superposition that makes reading a simultaneous projection across many features makes writing a sequential, direction-specific projection onto one
• IN directionality-as-universal-geometric-property
  Asymmetry (A→B ≠ B→A) appears independently in ROME's separate-edit constraint, Park's signed binary contrasts (ℓ̄_{w0⇒w1} = ℓ̄_{w1} − ℓ̄_{w0}), and SBERT's tied-weight siamese architecture, establishing directionality as an invariant geometric property of the superposed space rather than a per-implementation artifact.
• IN disambiguation-boundary-violation-is-universal-computational-constraint
  The need for semantic context to resolve disambiguation appears as a recurring pattern across processing hierarchies: both compilers (the C lexer hack requiring symbol table feedback across the lexer-parser boundary) and LLMs (prompt injection exploiting the inability to distinguish instructions from data) exhibit cases where lower-level processing cannot resolve meaning without higher-level semantic knowledge, suggesting that formal processing-level separations face inherent pressure from disambiguation demands that cross those boundaries.
• IN disambiguation-underlies-prompt-security-architecture
  The universal disambiguation constraint (requiring semantic context to cross processing boundaries) and the dual-use authority hierarchy (system prompts simultaneously enabling control and creating attack surfaces) are two manifestations of the same root limitation: LLMs cannot reliably maintain semantic boundaries between different types of input, whether instruction vs. data or system vs. user authority.
• IN disentanglement-vs-dictionary-learning
  Disentanglement assumes features ≤ dimensions (undercomplete basis), while dictionary learning/sparse coding assumes features > dimensions (overcomplete dictionary), aligning with compressed sensing and relevant to superposition.
• IN disentqa-two-answers-per-question
  DisentQA (Neeman et al., 2022) trains a model to output two answers per question—one from context and one from parametric memory—giving users agency to choose.
• IN distance-contrast-vanishes-in-high-dimensions
  In high-dimensional i.i.d. data, the ratio of farthest-to-nearest-neighbor distances converges to 1 (distance contrast → 0), undermining the discriminative power of k-NN and other distance-based algorithms.
• IN distilbert-95pct-performance-60pct-params
  DistilBERT (2019) retains 95% of BERT's benchmark performance with 60% of its parameters (66M vs 110M) via knowledge distillation
• IN distillation-and-routing-jointly-decouple-capability-from-cost
  Knowledge distillation (validated across the full scale spectrum, from 110M to 2T parameters) and multi-granularity compute routing (intra-model MoE and inter-model task routing) together decouple capability from inference cost along two independent axes: distillation compresses capability into smaller models, while routing allocates compute only where needed — together enabling deployment of frontier-class capability at a fraction of nominal cost.
• OUT distillation-could-enable-safe-capability-distribution
  Knowledge distillation's scale-invariant validation (from 110M to 2T parameters) combined with routing mechanisms that decouple capability from inference cost could enable safe capability distribution — deploying smaller, more controllable models that retain frontier knowledge at accessible cost — provided the information-theoretic inseparability of capability and vulnerability does not propagate through the distillation process itself.
• OUT distillation-enables-safe-capability-cascading
  Knowledge distillation's scale-invariant validation (from 110M DistilBERT to 2T Behemoth codistillation) combined with parameter redundancy evidence (95% performance at 60% parameters) enables safe capability cascading — frontier capabilities can systematically flow to smaller, more deployable models while maintaining quality.
• IN distillation-validated-across-full-scale-spectrum
  Knowledge distillation is validated as a scale-invariant capability across the full spectrum of language model sizes: from BERT-scale (DistilBERT retaining 95% performance at 60% of parameters) to frontier-scale (Llama 4 Maverick codistilled from the unreleased ~2T-parameter Behemoth), demonstrating that larger models reliably compress their capability into smaller ones regardless of absolute scale.
• OUT distributed-acquisition-bounds-editing-scope
  Knowledge editing is fundamentally a correction mechanism (fixing mis-encoded local associations) rather than a creation mechanism, because novel factual knowledge requires the distributed corpus exposure that a single rank-one update on one MLP layer cannot replicate.
• IN ditto-lopez-1992-motivated-skepticism
  Ditto & Lopez (1992) demonstrated motivated skepticism: people apply stricter evidential standards to arguments they dislike versus those they already favor.
• IN dl-ontologies-influence-conceptual-similarity
  d'Amato et al. (2008) demonstrated that Description-Logic ontologies influence conceptual similarity measurement in NLP.
• IN dola-dynamic-layer-selection
  DoLa uses a dynamic layer-selection strategy to choose appropriate premature/mature layers per token, distinguishing it from static contrastive decoding with fixed layer splits.
• IN dual-expertise-crisis-spans-deployment-and-security
  The LLM field faces a dual expertise crisis with a shared root cause: the expertise scalability paradox limits deployment practitioners while each capability advance widens the security expertise gap — both crises stem from inherently experiential knowledge barriers, yet compound each other because safe deployment requires both skill sets simultaneously in the same practitioners.
• OUT dual-layer-alignment-provides-resilient-safety-despite-individual-fragility
  The dual-layer alignment stack — with training-time alignment diversified into three independent paradigms and inference-time reasoning evolving from linear to branching structures — provides resilient safety coverage despite individual layer fragility, because failures in one layer or paradigm are caught by independent mechanisms at the other layer.
• OUT dual-layer-control-insufficient-without-injection-defense
  The dual-layer LLM control stack — training-time alignment diversity (RLHF, DPO, Constitutional AI) and inference-time prompting sophistication (CoT, self-consistency, context engineering) — would be sufficient for safe agentic deployment, but prompt injection's architectural vulnerability undermines the entire inference-time control layer.
• IN dual-vulnerability-mechanisms-compound-across-memory-timescales
  The agentic memory architecture likely faces compounding vulnerability across its temporal levels through two related mechanisms operating on overlapping timescale structures: memory security surfaces may expand at each level (potentially creating distinct attack vectors at intra-turn, inter-episode, and cross-session scales) while prompt fragility amplifies across those same levels (compounding existing instruction-data confusion through both wider context windows and persistent state), suggesting a combined vulnerability that may grow faster than either mechanism alone.
• IN economic-and-governance-failures-jointly-compound-irremediable-security-ratchet
  The irremediable security ratchet is compounded by two independent institutional failures operating on the same pool of already-released, unrecallable model weights: economic pressure accelerates the ratchet by driving wider, cheaper diffusion of capabilities that embed irretirable vulnerabilities, while governance failure amplifies it by ensuring no agreed framework constrains the resulting security debt — and neither failure mechanism has a plausible reversal pathway because both are structurally entrenched by the open-weight ecosystem's distribution dynamics.
• OUT economic-cost-decoupling-could-resolve-expertise-paradox
  Economic pressure decoupling capability from cost could resolve the expertise scalability paradox — making experiential deployment learning affordable and thus scalable beyond the current craft knowledge bottleneck — enabling a virtuous cycle where lower cost drives more deployment, which builds more expertise, which enables more responsible deployment.
• IN economic-efficiency-accelerates-irremediable-security-ratchet
  Economic pressure to decouple capability from cost accelerates the irremediable security ratchet: cheaper access drives wider diffusion of both innovation value and embedded vulnerabilities at legal and epistemic levels, while the cost reduction itself is practically irreversible — once affordable, capability cannot be re-contained, and each price reduction embeds more irretirable security debt in more hands.
• IN economic-pressure-drives-capability-cost-decoupling
  Exponential training cost scaling across model generations creates economic pressure that drives architectural innovations systematically decoupling capability from cost: MoE decouples parameter count from inference compute, knowledge distillation decouples model size from output quality, and compute routing selects granularity per task — all validated across the full scale spectrum from 110M to 2T parameters.
• OUT edit-complexity-is-geometrically-necessary
  The O(D²) cost of a rank-one knowledge edit is a fundamental lower bound imposed by the geometry of superposition, not an implementation artifact: any edit that preserves the covariance-geometric structure of the residual stream must operate in the whitened D-dimensional subspace, incurring at least O(D²) parameter modification
• OUT editable-semantic-space
  The residual stream, equipped with its covariance-geometric structure, constitutes a well-defined editable semantic space in which knowledge can be read (SAE feature activations, Park polytope coordinates) and written (ROME rank-one value-projection updates) as addressable, independently manipulable units.
• OUT editing-reliability-under-superposition
  Rank-one knowledge editing is reliable as a single-fact correction mechanism in superposed models because covariance whitening provides sufficient feature separation to isolate the target association from the background superposition, making the edit direction well-defined and locally confined.
• IN eh2022-path-patching-edge-intervention
  Path patching replaces the message sent from node u to node v along a computation-graph edge with the message produced on a different (reference) input to test whether that edge is causally necessary
• IN electra-replaced-token-detection-gan-inspired
  ELECTRA (2020) replaces MLM with replaced-token detection: a small generator creates plausible substitutions while a larger discriminator detects them, inspired by GANs
• OUT elhage-2022-publication-affiliation
  The toy models of superposition paper was published September 14, 2022, authored by Anthropic and Harvard, as part of the Transformer Circuits Thread.
• IN embedding-inversion-email-tweet-success-rates
  Translated embeddings enable extraction of ~80% of identifiable information from emails and ~67% from tweets for certain model pairs
• OUT engineering-maturity-sufficient-for-safe-agentic-deployment
  Engineering maturity — standardized training pipelines, reproducible alignment, defense-in-depth practices — provides a sufficient foundation for safe agentic AI deployment at scale, compensating for theoretical gaps through empirical rigor and layered defenses.
• IN engineering-maturity-systematically-outpaces-theoretical-understanding
  The LLM field is systematically characterized by engineering maturity outrunning theoretical understanding — standardized pipelines, reproducible results, and practical compensations consistently succeed at every level of the stack despite fundamental theoretical insufficiency that would normally preclude confidence.
• IN entityquestions-82pct-usable
  Only 82% of EntityQuestions questions are retained for evaluation because the remaining 18% lack unique Wikidata entity annotations.
• IN epiphenomenal-correctness-probe-vs-ablation
  Cross-model transfer probe accuracy for correctness prediction is 66% (exceeding 55% permutation and 62.9% majority-class baselines), yet full-subspace causal ablation produces only a 1.5% flip rate (5.5% under relaxed protocol), demonstrating shared correctness information is encoded but not causally deployed.
• IN epiphenomenal-correctness-transfer-vs-causal
  Shared information across LLMs is linearly decodable (66% transfer accuracy) but exerts minimal causal influence on predictions (1.5%–5.5% flip rate under ablation), demonstrating epiphenomenal correctness
• IN eps-mixture-index-inversely-correlated-irreducibility
  A lower ε-mixture index M(f) value indicates higher feature irreducibility, meaning the feature is less 'mixture'-like and more genuinely multi-dimensional.
• IN esmfold-10x-faster-than-alphafold2
  ESMFold runs approximately 10x faster than AlphaFold2 for protein structure prediction by using embeddings instead of multiple sequence alignments.
• OUT euclidean-partial-success-due-to-isotropic-covariance
  The partial functional success of Euclidean geometry in LLaMA-2's unembedding space is attributed to approximately isotropic covariance arising from weight initialization or implicit regularization, not a general guarantee transferable across architectures.
• OUT evaluation-geometry-is-editing-coordinate-system
  The optimal evaluation metric for an embedding model (cosine/Spearman pipeline) is simultaneously the optimal coordinate system for specifying knowledge edits, because both are readouts of the same universal covariance geometry
• OUT evaluation-geometry-predicts-editability
  The convergence of evaluation geometry (cosine/Spearman in SBERT/MTEB) and editing geometry (covariance whitening in ROME) on the same second-moment structure means that improving evaluation alignment and enabling reliable editing are two operational views of the same geometric optimization over the residual-stream covariance.
• OUT evaluation-geometry-valid-independent-of-aggregation
  The convergence of the cosine→Spearman evaluation pipeline across SBERT and MTEB validates the *metric itself* as a geometric probe of semantic similarity, independent of any structural bias in how per-task scores are aggregated into a final ranking.
• IN event-calculus-initiates-terminates-holdsat
  The event calculus uses predicates initiates(E, F, T), terminates(E, F, T), happensAt(E, T), and a single holdsAt inertia axiom to solve the frame problem.
• IN event-calculus-single-general-inertia-axiom
  The event calculus solves the frame problem using a single general inertia axiom (holdsAt) plus domain-specific initiates/terminates/happensAt facts, rather than per-fluent axioms.
• IN evt-max-bound-subgaussian
  The expected maximum similarity over M layer pairs is bounded by μ + 3σ√(log M) under sub-Gaussian tail conditions, requiring no independence assumptions
• IN expertise-crisis-compounded-by-invisible-capacity-inversion
  The dual expertise crisis — spanning both deployment and security — is compounded by the permanent invisibility of the capacity inversion between pretraining and alignment: practitioners cannot diagnose the structural flaw they need expertise to address because craft-based validation masks the very inversion that makes alignment structurally capacity-constrained relative to pretraining.
• IN expertise-crisis-entrenched-by-both-diffusion-and-invisibility
  The dual expertise crisis is entrenched by two independent compounding mechanisms: innovation diffusion spreads security challenges to institutions lacking expertise to manage them (external entrenchment), while the invisible capacity inversion prevents even expert practitioners from diagnosing the structural flaw they need to address (internal entrenchment).
• OUT expertise-paradox-resolvable-via-persistent-agentic-memory
  The expertise scalability paradox — where craft knowledge resists formalization and cannot scale with adoption — could be resolved by encoding deployment expertise in persistent agent memory rather than requiring it in every practitioner, effectively making the agentic paradigm the solution to its own expertise bottleneck.
• IN expertise-scalability-paradox
  The LLM field faces an expertise scalability paradox: the adoption flywheel demands exponentially more practitioners with deployment expertise, but that expertise is recursively experiential — it cannot be acquired faster than the rate of hands-on learning, creating a structural bottleneck that widens with every adoption cycle.
• OUT expertise-scalability-resolvable-by-agentic-ai-itself
  The expertise scalability paradox — where the adoption flywheel demands exponentially more practitioners with deployment expertise but craft knowledge doesn't scale through documentation — could be resolved by the very capability the field produces: agentic AI systems that augment less experienced practitioners' deployment expertise, enabling safe deployment with AI-assisted guidance rather than requiring hard-won experiential knowledge.
• IN expertise-security-dependency-entrenched-by-innovation-diffusion
  The vicious cycle between security and expertise (security mitigation requires deployment experience that creates exposure, expertise requires the deployment that the security gap makes dangerous) is further entrenched by innovation's dual-level uncontainability: since innovations diffuse across legal and epistemic boundaries regardless of intent, security challenges they generate cannot be restricted to institutions possessing the expertise to manage them.
• OUT fable-5-mythos-5-suspended-june-2026
  Both Fable 5 and Mythos 5 were suspended on June 12, 2026 by a US Department of Commerce directive.
• OUT fable-5-provides-public-mythos-class-access
  Fable 5 provides public access to Mythos-class model capability, serving as a safety-guardrailed version of the restricted Mythos line.
• IN feature-clumping-downstream-actions
  The most central explanation for feature clumping in activation space is similar downstream actions (output-effect similarity), not merely correlated input activations
• OUT feature-hierarchy-empirical-validation
  The hierarchical feature structure predicted geometrically by Park's orthogonality theorem (parent⊥child−parent) is empirically instantiated in SAE decoder-space neighborhoods: abstract/general features (transit infrastructure) subsume concrete/specific features (Golden Gate Bridge, Alcatraz), and this neighborhood hierarchy mirrors the WordNet synset hierarchy that Park validates across Gemma-2B and LLaMA-3-8B.
• OUT feature-level-editing-reliability
  SAE-identified features can serve as interpretable, causally-grounded targets for knowledge editing—specifying edits in semantic feature space rather than raw weight matrices—because the residual stream is the universal substrate and knowledge is locally stored, contingent on feature activations being causally meaningful rather than mere statistical correlates.
• OUT feature-neighborhood-as-geometric-theorem-instantiation
  SAE feature neighborhood structure (e.g., Golden Gate Bridge → San Francisco → California) is the concrete empirical instantiation of the covariance-geometric semantic space at the interpretable level: decoder-space proximity reflects the same subordination relations predicted by Park's orthogonality theorem, unifying interpretability with geometric theory.
• IN festinger-when-prophecy-fails-1956
  Festinger, Riecken & Schachter (1956) published 'When Prophecy Fails,' a classic field study demonstrating belief perseverance after a failed doomsday prediction by a cult.
• OUT few-shot-learning-introduced-gpt3
  Few-shot learning for language models was introduced with GPT-3 by Brown et al. (NeurIPS 2020, arXiv:2005.14165)
• IN field-cannot-predict-its-own-next-strengths
  The LLM field's development has been characterized by empirical discovery rather than theoretical prediction: the NLP evolution followed an engineering-driven progression, and even the field's most valuable structural properties (cross-boundary innovation, parameter redundancy) were discovered empirically rather than designed — this pattern of engineering maturity outpacing theoretical understanding suggests that systematic capability forecasting faces significant challenges.
• IN field-discovers-strengths-empirically-not-by-design
  The LLM field's most valuable structural properties — cross-boundary innovation driving transformation and parameter redundancy enabling reliability — were both discovered empirically rather than designed, reinforcing the systematic pattern of engineering maturity outpacing theoretical understanding from two independent directions.
• IN field-epistemic-closure-prevents-independent-safety-validation
  The LLM field exhibits complete epistemic closure that prevents independent safety validation: its craft methodology is unfalsifiably self-consistent (the strongest quantitative evidence — scaling laws, information-theoretic constants — validates the empirical approach that generated it), AND its primary safety evaluation mechanism is doubly circular and vulnerable (alignment is produced by the methodology it compensates for, evaluated by a reward model inheriting the paradigm's vulnerabilities) — meaning neither the methodology nor its safety assurances admit external validation from within the field's own epistemic framework.
• IN file-drawer-effect-publication-bias
  The file drawer effect refers to the discarding of data that conflicts with experimenter expectations as 'unreliable,' resulting in a skewed published record and constituting a form of publication bias.
• IN first-machine-generated-book-1984-racter-rule-based
  The first machine-generated book was produced in 1984 using Racter (a rule-based system); the first neural-network novel (*1 the Road*, 60M words) appeared in 2018, and the first machine-generated science book (Beta Writer, *Lithium-Ion Batteries*, Springer) in 2019.
• OUT five-defense-dimensions-adequate-for-agentic-reliability
  The five independent LLM defense dimensions — training-time alignment, inference-time prompting, data integrity, architectural hardening, and monitoring — provide adequate reliability for large-scale agentic deployment when all dimensions are simultaneously maintained and the three convergent capabilities (context windows, alignment, efficiency) are in place.
• IN five-open-source-models-in-experiments
  Five additional open-source LLMs were used alongside the three closed-source models in the experiments, with specifics detailed in Appendix B.
• OUT five-samples-per-dimension-minimum-rule
  A commonly cited rule of thumb requires at least 5 training examples per dimension (feature) for reliable generalization in high-dimensional data.
• IN five-samples-per-dimension-ml-rule-of-thumb
  A frequently cited practical rule of thumb in machine learning requires at least 5 training examples per dimension for reliable generalization.
• IN fjlt-sub-quadratic-projection-time
  The Fast Johnson-Lindenstrauss Transform (Ailon & Chazelle 2006) computes a JL projection in O(d log d + k^{2+γ}) time using a structured matrix of permutation, subsampling, and Hadamard transforms, versus O(kd) for naive matrix-vector multiplication.
• OUT float16-parameter-size
  In float16 precision, each parameter uses 2 bytes, so 1 billion parameters equals 2 GB of storage
• IN float16-parameter-size-2bytes
  One float16 parameter requires 2 bytes of storage, so 1 billion float16 parameters require approximately 2 GB.
• IN float16-two-bytes-per-parameter-one-b-approx-2gb
  Float16 uses 2 bytes per parameter, so a 1B-parameter model occupies approximately 2 GB of memory in float16 precision.
• IN fluent-calculus-reifies-states-as-terms
  The fluent calculus reifies state predicates into terms (e.g., open ∘ on), treating a state as an object rather than a set of true predicates, and actions are described as transformations between state terms.
• IN fluent-definition-time-dependent-predicate
  A fluent is a time-dependent predicate (e.g., open(t), on(t)) representing a condition that can change over time, as opposed to a static proposition.
• IN fluent-occlusion-sandewall-occlude-predicate
  Fluent occlusion, proposed by Sandewall, introduces auxiliary predicates like occludeopen(t) representing 'permission to change'; a fluent may change only if its occlusion predicate is true at the next time point.
• IN formal-understanding-insufficient-across-llm-stack
  Formal theoretical understanding consistently proves insufficient across the entire LLM stack: RLHF's complete mathematical specification fails without dozens of engineering details, prompting's irreducible sensitivity resists formal analysis, the capacity bottleneck inverts between pretraining and alignment stages — and the inversion means that even a correct scaling theory for one stage actively misleads for the next.
• IN formal-verification-impossibility-extends-to-disambiguation-layer
  The craft discipline's formal verification impossibility (grounded in NLP's AI-complete classification) extends to the disambiguation layer where the most critical security vulnerability operates: since prompt injection exploits the universal disambiguation problem (semantic context crossing processing boundaries), the formal verification deficit is rooted at the most fundamental processing level — not merely at the surface of deployment complexity.
• IN formal-verification-impossibility-makes-inference-vulnerability-unresolvable
  The inference information architecture's joint vulnerability — context engineering and retrieval augmentation both ceiling-limited by inference control — is formally unresolvable: the disambiguation layer where prompt injection originates is the same layer where formal verification is impossible due to NLP's AI-complete classification, placing inference security provably beyond the reach of formal methods.
• IN formal-verification-impossible-given-ai-completeness
  The craft discipline's inability to achieve formal safety verification may reflect a fundamental impossibility rather than a maturity gap: NLP's classification as AI-complete implies that formally verifying NLP system behavior requires solving the full AI problem, explaining why safety assurance remains fundamentally informal despite decades of engineering maturity and massive investment.
• IN foundation-model-term-stanford-2021
  The term 'foundation model' was coined by Stanford's Center for Research on Foundation Models (CRFM), established August 2021, for models trained on broad data and adaptable to many tasks
• IN frame-problem-origin-mccarthy-hayes-1969
  The frame problem was defined by John McCarthy and Patrick Hayes in their 1969 paper 'Some Philosophical Problems from the Standpoint of Artificial Intelligence'.
• IN frame-problem-paper-machine-intelligence-vol4
  McCarthy and Hayes' 1969 frame problem paper was published in Machine Intelligence, Vol. 4.
• IN frame-solution-categorization-action-vs-condition-centered
  Fluent occlusion and predicate completion solutions are action-centered; successor state axioms are condition-centered (one per fluent); fluent and event calculus solutions are state-term-centered.
• IN frobenius-norm-feature-count-proxy
  The Frobenius norm ||W||^2_F = sum_i ||W_i||^2 is used as a basis-independent proxy for the number of learned features, since ||W_i||^2 ≈ 1 when a feature is represented and ≈ 0 otherwise.
• IN frontier-accessibility-gap-persists-despite-capability-convergence
  Frontier competition drives capability parity between proprietary and open-weight models, but safety classification and licensing restrictions independently constrain which capabilities can be widely deployed, creating a persistent accessibility gap that widens as capabilities increase.
• IN frontier-agentic-convergence-demands-alignment-diversity
  As frontier models converge on multimodal agentic capabilities, alignment has concurrently diversified into three independent paradigms (RLHF, DPO family, Constitutional AI), a coincidence that may prove relevant if different alignment approaches turn out to offer distinct advantages for varied deployment contexts.
• IN frontier-capability-and-deployment-accessibility-diverge
  Frontier competition drives capability advancement across both proprietary and open-weight models, while practical deployment benefits from optimization across multiple complementary levels (architectural, algorithmic, memory, compute-aware), suggesting that realizing frontier capabilities in production involves substantial engineering effort beyond model training alone.
• IN frontier-deployment-epistemically-impossible-under-current-expertise-structure
  Frontier deployment faces a compounding structural barrier: the triple bind (speed, ignorance, circular dependency) is unlikely to be resolved simply by growing the practitioner pool because the dual expertise crisis makes it difficult to produce the required specialists at the rate demanded — deployment expertise requires deployment experience that requires deployment expertise, while security expertise gaps tend to widen with each capability advance. These dynamics suggest that scaling practitioners alone is insufficient, though they do not establish that resolution is impossible through other structural interventions.
• IN frontier-deployment-faces-maximal-expertise-gap
  Frontier capability deployment faces a maximal expertise gap: reliable deployment demands irreducible operational expertise, but the innovation frontier — where new capabilities appear — is precisely where formal understanding is weakest, forcing practitioners to develop expertise through trial-and-error with the least-understood systems.
• IN frontier-deployment-paradox-fastest-adoption-with-least-expertise
  The most advanced capabilities reach production fastest via the adoption flywheel, yet the frontier expertise gap ensures they arrive with maximal security exposure and minimal expertise to manage it — a systematic inversion between deployment velocity and deployment readiness that worsens with each capability generation.
• IN frontier-deployment-structurally-impossible-and-epistemically-invisible
  Frontier deployment faces a compounding structural challenge: the triple bind is unlikely to be resolved by scaling practitioners alone (because the dual expertise crisis makes producing them difficult at the required rate), and a primary structural factor contributing to this difficulty — capacity inversion between pretraining and alignment — tends to be hidden beneath craft validation. This creates a failure mode where the system struggles to succeed while a key reason for that struggle remains difficult to perceive, though this does not rule out resolution through other structural interventions.
• IN frontier-deployment-triple-bind
  Frontier deployment faces a triple bind of speed, ignorance, and circular dependency: the adoption flywheel pushes frontier capabilities to production fastest, the expertise gap ensures they arrive with minimal operational knowledge, and the security mitigation experience needed can only be acquired through the very deployment that creates the exposure — all three forces converge maximally at the innovation frontier.
• IN frontier-expertise-gap-confronts-unpredictable-capabilities
  Frontier deployment faces a maximally intractable expertise challenge: the expertise gap is widest at the innovation frontier (where formal understanding is weakest and deployment demands are highest) AND the capabilities requiring that expertise cannot be predicted in advance — making it impossible to pre-train practitioners for the capabilities they will need to deploy, even if the experiential learning barriers could somehow be overcome.
• IN ft-fine-tuning-catastrophic-multihop-failure
  Fine-tuning (FT) on layer 21/31 yields 0–2.8% multi-hop accuracy compared to approximately 40% for the unedited base model.
• OUT fv-cyclic-impossibility-proof
  No non-zero constant vector offset can implement a cyclic mapping because w + v = w' and w' + v = w algebraically forces v = 0 and w = w', yet a single FV does produce cyclic behavior (e.g., antonym lookup).
• OUT fv-reconstruction-from-vocabulary-insufficient
  Reconstructing a vector to match all 50k decoded vocabulary tokens of the true FV (KL ≈ 0.01) still yields substantially lower accuracy (e.g., Antonym: 39.6% vs 48.2%; Capitalize: 51.5% vs 70.5%), proving FVs encode information beyond output-token distributions.
• IN gdpp-eigenvalue-transformation-reduces-condition-number
  GD++ transforms the loss Hessian eigenvalues from λᵢ to λᵢ − 2γλᵢ² + γ²λᵢ³ via the input transformation xᵢ ← (I − γXXᵀ)xᵢ, reducing the condition number κ = λ₁/λₙ; optimal γ ≈ 0.099 for N=25 and γ ≈ 0.029 for N=100.
• IN gelfond-lifschitz-action-languages-1993-1998
  Gelfond and Lifschitz proposed Action Languages (B, C, E) in 1993 and 1998 as a logic-programming approach to the frame problem.
• IN gelu-multiplication-identity-enables-mlp-arithmetic
  The identity √(π/2)·(GeLU(x+y) − GeLU(x) − GeLU(y)) = xy + O(x³+y³) enables elementwise multiplication inside a transformer's MLP, forming the key mechanism for performing nonlinear arithmetic within the model.
• IN gemma-2b-architecture-specs
  Gemma-2B has 2 billion parameters, was pre-trained on 3 trillion tokens, uses a 256K vocabulary, and has a 2,048-dimensional representation space.
• IN gene-ontology-three-sub-ontologies
  Gene Ontology consists of three sub-ontologies: Biological Process (BP), Molecular Function (MF), and Cellular Component (CC); semantic similarity is computed per ontology then aggregated.
• IN generality-and-alignment-create-compounding-adoption-flywheel
  The Transformer's demonstrated cross-domain generality and alignment's role in igniting mass adoption appear to form a compounding dynamic: alignment enabled adoption (as ChatGPT demonstrated), adoption funded capability expansion into new domains, and the Transformer's architectural flexibility allowed those capabilities to generalize — suggesting these two properties reinforce each other, though the specific causal links between cross-domain success, investment flows, and alignment research funding remain underspecified by the evidence.
• OUT generation-gap-pre-post-decision-cka
  Pre-decision (input processing) representations align strongly (CKA = 0.875) while post-decision representations diverge sharply (CKA = 0.274), indicating convergence reflects shared input-processing constraints rather than shared reasoning strategies
• OUT geometric-closed-loop-eval-edit-navigate
  The MTEB evaluation coordinate system, ROME's rank-one editing, and SAE feature navigation form a single closed geometric loop: evaluation identifies the whitened directions to read, ROME modifies one whitened direction to write, and SAE neighborhood traversal navigates between whitened directions—each operation is a different linear functional on the same covariance-geometric space, unified by the Riesz map.
• OUT geometric-convergence-as-mathematical-attractor
  The cross-model universality of feature geometry combined with its ontological status as a model-independent semantic object implies LLMs are converging to a shared mathematical attractor: the covariance/whitening geometry is the unique fixed point that any differentiable language model must instantiate, not an architectural artifact.
• OUT geometric-editing-addressability-bound
  The covariance-geometry framework defines a precise and minimal addressable space for knowledge editing (rank-one updates to a single MLP value projection), but the combination of superposition and distributed corpus acquisition structurally bounds this to single-fact local corrections—edits cannot create novel multi-hop associations because the target knowledge was never locally consolidated in the first place.
• OUT geometric-framework-not-artifact
  The full geometric framework of LLMs (covariance whitening, polytope decomposition, Riesz isomorphism, cross-model convergence) reflects genuine architectural structure rather than a mathematical artifact of the analysis method, provided the underlying orthogonality is not reducible to trivial set-inclusion between parent and child token sets.
• OUT geometry-as-universal-semantic-currency
  The covariance geometry is the single operational definition of "meaning" in LLMs, simultaneously determining what can be measured (evaluation via cosine/Spearman), what can be modified (rank-one editing via C⁻¹k*), what converges across architectures (SAE/Park universality), and what is hierarchically structured (feature neighborhoods as theorem instantiations)—making it a model-independent semantic currency rather than an architecture-specific artifact.
• OUT geometry-ontological-status
  The covariance/whitening geometry is not merely a convenient analytical tool but possesses ontological status as a genuine model-independent semantic structure, because three independent lines converge: it is the operational metric for editing and interpretation (depth-3), it is universal across architectures (depth-2), and it converges with externally-validated human-judgment metrics (depth-3).
• IN georgetown-experiment-1954
  The Georgetown experiment (1954) was the first automatic translation demonstration (Russian to English); the ALPAC report (1966) subsequently cut machine translation funding
• IN georgetown-ibm-1954-alpac-1966-funding-freeze
  The Georgetown-IBM experiment (1954) achieved the first automatic translation of 60+ Russian-to-English sentences, and the over-optimistic claims that followed led to the ALPAC report (1966) and a subsequent funding freeze on machine translation research
• IN google-ai-research-split-brain-deepmind
  Google's AI research is organized across Google Brain and Google DeepMind as separate research divisions under the broader Google AI umbrella within Alphabet Inc.
• IN google-paradox-is-instance-of-craft-innovation-uncontainability
  Google's paradox — inventing and validating the Transformer and BERT at web scale only to see them become universal infrastructure powering competitors — is a specific instance of the craft discipline's general innovation uncontainability: any institution that develops and publishes craft knowledge inherently exports the conditions for others to replicate and extend it, making Google's experience predictable rather than exceptional.
• IN governance-and-deployment-capacity-jointly-outpaced-by-diffusion
  The open-weight ecosystem faces a double outpacing dynamic: governance failure amplifies irretirable security debt (no agreed standard for "open," weights cannot be recalled once released) while responsible deployment capacity structurally lags behind capability diffusion (expertise is experiential and unscalable), ensuring both regulatory and operational safeguards are permanently behind the diffusion frontier along independent dimensions.
• OUT gplvm-latent-to-data-opposite-of-kpca
  GP-LVM (Lawrence, 2005) maps from latent space to data space using a Gaussian Process kernel, which is the opposite direction from Kernel PCA that maps from data to latent space, and GP-LVM provides uncertainty estimates for predictions
• OUT gpt-decoder-only-architecture
  GPT models use a decoder-only transformer architecture, not the full encoder-decoder transformer.
• OUT gpt-decoder-only-autoregressive
  GPT models are decoder-only Transformers that use causal (autoregressive) language modeling, predicting each token sequentially given all prior tokens.
• OUT gpt-trademark-descriptive-generic-2023
  The USPTO ruled 'GPT' was both descriptive and generic in May 2023, challenging OpenAI's trademark claim.
• OUT gpt-uses-decoder-only-transformer
  GPT models use only the decoder portion of the transformer architecture, not the encoder
• OUT gpt3-175b-parameters-few-shot-brown-2020
  GPT-3 (Brown et al., 2020) has 175 billion parameters and demonstrated few-shot in-context learning without gradient updates.
• OUT gpt3-175b-params-few-shot
  GPT-3 has 175 billion parameters (released May 2020) and introduced few-shot in-context learning
• OUT gpt3-175b-params-few-shot-zero-shot-may-2020
  GPT-3 (May 2020) had 175 billion parameters and introduced practical few-shot and zero-shot in-context learning without task-specific fine-tuning.
• OUT gpt4-released-march-2023-multimodal
  GPT-4 was released March 14, 2023 and is multimodal (text + image input, text output)
• OUT gpt5-released-august-2025-all-chatgpt-users
  GPT-5 was released in August 2025 and made available to all ChatGPT users as the latest generation in the GPT family.
• IN gradient-masking-sparse-relu-networks
  Standard gradient-based adversarial attacks fail on sparse ReLU networks because ~99% of ReLU neurons are in the zero regime, effectively zeroing out gradients (gradient masking).
• IN graph-based-nldr-cannot-embed-out-of-sample
  Most graph-based NLDR methods (LLE, Hessian LLE, Isomap, MVU, Laplacian Eigenmaps, LTSA) cannot natively embed out-of-sample points and require additional regularization (e.g., RKHS) to do so, whereas KPCA and Autoencoders have an internal model that permits new-point mapping.
• OUT grokking-memorization-phase-manageable-under-controlled-training
  Grokking's memorize-then-generalize dynamic implies that the security-vulnerable memorization phase is a transient training state that resolves under continued training — models move from memorization (maximal data extractability) to generalization (compressed, abstract representations), making the vulnerability window manageable under controlled training conditions where intermediate checkpoints are secured.
• IN grokking-memorize-then-generalize
  Grokking is the phenomenon where a model first memorizes training data (overfitting), then suddenly learns the underlying algorithm and generalizes, discovered via mechanistic interpretability of modular arithmetic models.
• IN group-theoretic-irreducibility-equivalence
  When a task defines a finite group multiplication, the paper's reducibility definition (tensor-product decomposition of representation spaces) coincides with the standard group-theoretic definition of irreducible representations.
• IN hand-constructed-weights-not-trained
  The weight matrices in Akyürek et al. (2022) and von Oswald et al. (2023) are analytically designed (hand-constructed) and not produced by any optimization or training procedure.
• IN hanks-mcdermott-yale-shooting-objections
  Hanks and McDermott raised the Yale shooting problem as an objection to the default logic solution of the frame problem.
• OUT head-tail-divide-scale-validated
  The head-tail geometric divide (parametric vs. contextual knowledge split) is a genuine architectural property that generalizes across model scales (334M→20B) for single-hop factual knowledge, but its extension to multi-hop compositional queries is not yet validated.
• OUT head-tail-geometric-divide
  The parametric/contextual knowledge split is geometrically grounded rather than merely frequency-driven: head-of-distribution facts occupy individually addressable directions in the covariance-whitened feature space (enabling rank-one editing), while long-tail facts are distributed across superposed features where no single direction isolates the knowledge, making external retrieval the only faithful access mechanism.
• IN hella-swag-incorrect-options-sampled-from-lm
  HellaSwag's incorrect answer options in its multiple-choice video-description completion task are generated by sampling from a language model, making them plausible to LMs but trivial for humans to distinguish from the correct option
• IN helm-benchmark-evaluates-50-plus-scenarios-multiple-dimensions
  Stanford CRFM's HELM benchmark evaluates language models across 50+ scenarios spanning accuracy, calibration, robustness, fairness, efficiency, toxicity, and bias.
• IN helm-multi-metric-benchmark
  HELM (Holistic Evaluation of Language Models) from Stanford CRFM evaluates models across accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency
• OUT hierarchical-orthogonality-genuine-structure
  The observed parent⊥(child−parent) orthogonality in LLM representation spaces reflects genuine hierarchical semantics rather than a trivial set-inclusion artifact.
• IN high-confidence-oppositional-seeking
  High-confidence individuals more readily seek contradictory information to counterargue (oppositional news consumption), while low-confidence individuals prefer confirming information.
• IN high-confidence-reduces-myside-bias
  High confidence decreases myside bias because confident individuals seek opposing information to counterargue, while low confidence increases confirmation bias.
• IN high-dim-distance-concentration-ratio-approaches-unity
  In high-dimensional Euclidean space with i.i.d. assumptions, the ratio of the maximum to minimum pairwise distances between random points approaches 1 as d → ∞, making nearest-neighbor distinctions meaningless.
• IN highest-value-innovations-carry-highest-security-debt
  The LLM field's most transformative innovations carry the highest security debt: innovations that cross disciplinary and organizational boundaries — where transformative value is highest — arrive precisely at the frontier where formal security understanding is weakest.
• IN hop-depth-decreases-accuracy-all-methods
  Across all editing methods tested (Base, FT, MEND, ROME, MEMIT), 2-hop accuracy is substantially higher than 3-hop, and 3-hop higher than 4-hop, indicating multi-hop depth is the primary performance differentiator.
• IN hub-points-radovanovic-2010-jmlr
  Radovanović, Nanopoulos & Ivanović (2010, JMLR) demonstrated that in high-dimensional spaces, a small set of 'hub' points appear disproportionately in the k-NN neighbor lists of many other points, distorting classification, clustering, and retrieval.
• IN hubness-skews-knn-indegree-in-high-dimensions
  In k-NN digraphs in high dimensions, the indegree distribution becomes skewed such that a small set of 'hub' points appear disproportionately in neighbor lists, distorting classification and clustering.
• IN hughes-peaking-phenomenon-non-monotonic-accuracy
  The Hughes/peaking phenomenon (Hughes 1968; Zollanvari et al. 2020) states that with a fixed training sample size, classifier predictive power first increases with added dimensions, then degrades beyond a critical dimensionality—accuracy is non-monotonic in dimensionality.
• IN hungarian-discrete-vs-soft-ot-metrics
  The Hungarian algorithm produces discrete matchings evaluated only on Top-1 Accuracy, while soft OT methods (EMD, Sinkhorn, Gromov-Wasserstein) produce fractional transport plans evaluated on Top-1 Accuracy, Mean Rank, and Mean Cosine Similarity
• IN hypercube-sampling-10d-requires-10to20-points
  Sampling a 10-dimensional unit hypercube at spacing 10⁻² requires 10²⁰ points, compared to 10² for a 1-D interval—a factor of 10¹⁸ increase—illustrating the exponential growth of required samples with dimensionality.
• OUT hypersphere-to-hypercube-volume-ratio-approaches-zero
  The volume of the inscribed hypersphere relative to a d-dimensional unit hypercube follows V_sphere/V_cube = π^(d/2) / (d · 2^(d−1) · Γ(d/2)) → 0 as d → ∞.
• IN hypersphere-to-hypercube-volume-ratio-vanishes
  The ratio of hypersphere volume to hypercube volume, π^(d/2)/(d·2^(d-1)·Γ(d/2)), approaches 0 as d→∞, meaning in high dimensions almost all cube volume lies near the surface of a sphere of radius √(d/3).
• OUT icl-distinguishability-condition-kl-threshold
  The distinguishability condition (Eq. 14) requires that the sum over k tokens of KL divergence KL_j(θ*∥θ) exceeds ε^start_θ + ε^delim_θ for all θ ≠ θ*, where ε^delim_θ = 2(log c₂ − log c₁) + log c₄ − log c₃ and ε^start_θ = log(1/c₈).
• OUT icl-gd-gap-constant-across-scale
  The ICL–GD gap remains roughly constant across model sizes from 1.5B to 7B parameters, suggesting the phenomenon is architectural rather than a scaling artifact.
• OUT icl-gd-gap-persists-across-model-scale
  The behavioral gap between ICL and GD does not significantly narrow when model size increases from 1.5B (GPT2-XL) to 7B (LLaMA) on AGNews with N=8 demonstrations.
• OUT icl-gd-step-requires-od-hidden-space
  Theorem 1 in Akyurek et al. (2023) proves that a transformer with constant depth and O(d) hidden size can compute one gradient descent step on the linear regression objective, and N steps require N× the layers.
• IN illusory-correlation-positive-positive-cell
  Illusory correlation is driven by over-weighting the positive-positive cell (both events present) and neglecting the other three cells of a 2×2 contingency table, as demonstrated in the arthritis-weather study where actual correlation was approximately zero.
• OUT in-context-learning-reliably-replaces-finetuning
  In-context learning reliably replaces per-task fine-tuning for most downstream applications, as demonstrated by GPT-3's few-shot capabilities operating without gradient updates.
• OUT inference-compute-reliably-improves-reasoning
  Self-consistency (sampling multiple chain-of-thought paths and selecting by majority vote) and Tree of Thoughts (branching reasoning with backtracking) independently demonstrate that LLM reasoning reliability improves with inference-time compute investment — establishing a general compute-for-quality tradeoff at inference time — unless prompt sensitivity means each individual reasoning path is itself unreliable enough to undermine the ensemble.
• IN inference-control-degradation-compounds-structural-safety-deficit
  The structural safety deficit is compounded by inference-time control's self-undermining nature: as context windows expand to enable the agentic paradigm, the prompt authority hierarchy that provides runtime security simultaneously becomes a larger attack surface — the mechanism the field relies on for runtime safety guarantees structurally weakens as capabilities advance.
• IN inference-control-unimprovable-while-memory-surfaces-multiply
  The inference-time control layer is structurally unimprovable (resistant to formalization due to the systematic theory gap in prompt engineering) while its attack surface is actively multiplying (dual vulnerability mechanisms compounding across all memory timescales), producing a defense that gets weaker in precisely the dimension it cannot strengthen.
• OUT inference-democratization-enables-safe-distributed-deployment
  Democratized inference — CPU-only execution and single-executable distribution eliminating hardware and installation barriers — enables safe distributed deployment of capable models, lowering the expertise threshold for responsible use.
• IN inference-information-architecture-jointly-constrained-and-vulnerable
  The entire inference-time information architecture is compromised along two independent axes: context engineering and retrieval augmentation are jointly ceiling-limited by the structural degradation of the control surface they operate on (prompt fragility, injection vulnerability, theory gaps), while the complete memory hierarchy (bridging 10,000x horizontal capacity expansion and vertical cross-session persistence) creates a parallel complete vulnerability hierarchy expanding across all temporal levels — the information substrate enabling agentic capabilities is simultaneously constrained in what it can deliver and vulnerable in how it can be attacked.
• IN inference-time-control-is-structurally-self-undermining
  The inference-time control layer is structurally self-undermining at scale: prompt fragility compounds as context windows expand while the authority hierarchy that enables security simultaneously provides the attack surface that injection exploits — the two mechanisms meant to protect inference-time behavior actively erode each other as capability grows.
• IN information-content-defined-as-negative-log-probability
  Information Content of a concept x is defined as IC(x) = −log P(x), meaning more specific (less probable) concepts have higher information content.
• IN innovation-diffusion-creates-irremediable-security-ratchet
  LLM innovation creates an irremediable security ratchet: innovation value diffuses uncontainably at both legal and epistemic levels (foundational patents face genericness challenges, craft knowledge crosses all boundaries), while weight diffusion makes both innovation and vulnerability symmetrically uncontainable after release — each innovation cycle permanently expands the attack surface through a mechanism that cannot be reversed.
• IN innovation-frontier-is-security-blind-spot
  The LLM innovation frontier is inherently a security blind spot: innovation velocity peaks where formal understanding is weakest, and security challenges compound across all maturity dimensions, so the fastest-moving areas of the field are precisely those least equipped to address the security surfaces they create.
• IN innovation-transcends-organizational-and-disciplinary-boundaries
  The NLP revolution's most transformative contributions defy institutional ownership at two levels: organizationally, Google's inventions (Transformer, BERT, CoT) became the universal foundation powering every competitor; disciplinarily, techniques imported from robotics (RLHF), machine translation (attention), and compression theory (scaling validation) proved more impactful than NLP-native innovations.
• IN innovation-uncontainability-operates-at-legal-and-epistemic-levels
  LLM innovation is uncontainable at two independent and mutually reinforcing levels: legal/IP mechanisms fail (Transformer patent faces genericness challenges, GPT trademark deemed descriptive) AND craft knowledge transfer inherently crosses organizational boundaries — even if IP protections were effective, the knowledge itself would still diffuse through the field's empirical, experiential methodology.
• IN innovation-value-correlates-with-boundary-crossings
  The NLP revolution's most transformative contributions share a pattern of boundary-crossing: techniques imported from outside NLP (attention from machine translation, RLHF from Atari/robotics) became foundational, the resulting Transformer architecture exported to domains like protein folding, chess, and reinforcement learning, and organizationally, Google's inventions powered competitors — suggesting that crossing disciplinary and institutional boundaries is a strong indicator of innovation impact.
• IN innovation-velocity-peaks-where-formal-understanding-is-weakest
  The shift of the LLM innovation frontier from settled macro-architecture to actively contested micro-architecture configuration is consistent with the pattern of engineering maturity outpacing theoretical understanding — practitioners appear to concentrate innovation on components (activation functions, normalization, positional encoding) where empirical tuning succeeds but formal prescriptions remain absent, suggesting a tendency for higher-velocity innovation to occur where formal guidance is weakest.
• IN intervention-linearly-shifts-target-logit
  Adding α·λ̄_W to the model's representation (λ_C,α(x) = λ(x) + α·λ̄_W) with α ∈ [0, 0.4] linearly increases the target concept's logit while leaving causally separable concept logits unchanged; at α=0.4 in the 'Long live the' / male⇒female experiment, 'king' fell entirely out of the top-5 predictions.
• OUT inversion-and-drop-as-dual-boundary
  Kandpal's retrieval inversion (parametric accuracy *decreases* with document count while contextual accuracy *increases*) and ROME's multi-hop drop (40.5%→7.4% after editing) are the same geometric boundary observed from the read and write sides respectively: the rank-one addressable subspace fills the head directions parametrically but is silent on the tail, and the context window fills exactly those missing tail directions.
• IN ipo-quadratic-loss-no-bradley-terry
  Identity Preference Optimization (IPO) modifies DPO by using a quadratic loss instead of log-sigmoid and does not assume Bradley-Terry preferences, reducing overfitting on noisy preference data
• IN irreducibility-requires-both-tests
  A feature is classified as irreducible only if both S(f) is high (high mutual information across projections) and M_ε(f) is low (low geometric concentration in a strip); either condition alone is insufficient.
• IN irreducible-reducible-empirical-thresholds
  Empirically, irreducible features show S(f) ≈ 2.7 bits and M_ε ≈ 0.18, while reducible features show S(f) ≈ 0.37 bits and M_ε ≈ 0.64.
• IN jiang-conrath-1997-distance-formula
  Jiang & Conrath (1997) define semantic distance as dist(A,B) = IC(A) + IC(B) − 2·IC(LCS), converting it to a similarity score.
• OUT jl-lemma-chi-square-concentration-proof
  The classical JL proof uses the fact that for A ~ 𝒩(0,1)^{k×n}, the ratio ‖Ax‖²/‖x‖² follows a χ²(k) distribution, yielding the concentration bound Pr(r ∈ (1±ε)·k) ≥ 1 − 2·exp(−(k/2)·(ε²/2 − ε³/3)).
• OUT kalai-2023-good-turing-concentration-bound
  The Good-Turing concentration bound establishes that |p(U) − dMF| ≤ 3√(ln(4/δ)/n) with probability ≥ 1−δ, so the monofact estimate concentrates around the true missing-facts rate at rate O(1/√n).
• OUT kandpal-2023-bloom-parameter-range
  The long-tail knowledge study tested BLOOM models ranging from 560M to 176B parameters.
• OUT kandpal-2023-gpt-neo-model-sizes-tested
  GPT-Neo model sizes tested in Kandpal et al. (2023) include 125M, 1.3B, 2.7B, 6B, and 20B parameters.
• IN klayman-ha-1987-informational-value
  Klayman & Ha (1987) demonstrated that confirmation bias is not solely motivational; informational value and base-rate considerations also influence hypothesis-testing search strategy.
• IN knn-null-baseline-k-over-n-minus-1
  Under H₀ (independence), the expected mKNN similarity is E[mKNN(X,Y)] = k/(n−1), derived from the hypergeometric distribution of two uniform random k-subsets of an (n−1)-set with expected intersection k²/(n−1)
• IN knowledge-conflicts-survey-companion-repo
  The companion repository for the Xu et al. knowledge conflicts survey is hosted at https://github.com/pillowsofwind/Knowledge-Conflicts-Survey.
• OUT knowledge-lifecycle-closed-loop
  The full knowledge lifecycle in LLMs—distributed corpus acquisition → local MLP key-value storage → parametric recall → rank-one editing correction—forms a closed loop unified by the covariance geometry, where each stage is addressable in the same second-moment coordinate space.
• OUT knowledge-routing-architecture
  The LLM implements a natural knowledge-routing architecture: well-supported facts are parametrically stored in local MLP key-value memory (head of the distribution), while rare/long-tail facts are architecturally routed through the contextual channel (retrieval, extended context windows), with the routing boundary determined by corpus document frequency rather than an arbitrary design choice.
• OUT knowledge-routing-as-geometric-gate
  The parametric/contextual knowledge routing is a geometric gate based on subspace membership rather than a statistical frequency heuristic: the model routes to parametric recall when the fact direction lies within the rank-one addressable subspace, and to contextual retrieval when it does not, with the faithfulness validation (accuracy scaling with document relevance) as the empirical confirmation that the gate responds to geometry, not statistics.
• OUT knowledge-routing-faithfulness-validated
  The parametric/contextual two-channel knowledge architecture is a genuine computational duality rather than a surface-level re-ranking bias, because retrieval context demonstrably inverts the parametric accuracy trend (accuracy increases with document relevance for rare facts) and BM25 recall remains robust independently of parametric scaling.
• IN kv-caching-avoids-recomputation-autoregressive
  KV caching stores previously computed key and value vectors during autoregressive inference to avoid recomputation; prefilling is the initial forward pass that populates the cache.
• IN layernorm-applied-per-vector-not-batch
  Layer normalization (LayerNorm) is applied individually to each vector (per-position), not across the batch or sequence dimension.
• IN lcs-definition-most-specific-ancestor
  The Lowest Common Subsumer (LCS) of two concepts is the most specific shared ancestor in a taxonomy; e.g., for 'cat' and 'dog', the LCS is 'mammal' rather than 'animal'.
• IN lda-canonical-topic-modeling
  Latent Dirichlet Allocation (LDA) is the canonical topic modeling algorithm
• IN lemmatization-dictionary-stemming-rules
  Lemmatization uses a dictionary to map words to base forms and returns valid words; stemming uses rules to strip affixes and may produce non-words
• OUT levi-harper-identities-connect-revision-contraction
  The Levi Identity (K * P = (K − ¬P) + P) and Harper Identity (K − P = K ∩ (K * ¬P)) bidirectionally connect belief revision and belief contraction within the AGM framework
• IN lexer-first-phase-compiler-frontend
  Lexical analysis (lexing) is the first phase of a compiler frontend, converting a raw character stream into a sequence of tokens.
• IN lexer-fsm-parser-pushdown-automaton
  In compiler design, the lexer uses a finite-state machine (FSM) while the parser uses a pushdown automaton (stack-based); regular expressions cannot count or match recursive structures.
• OUT lexer-maximal-munch-rule
  The maximal munch (longest match) rule is the standard strategy in lexical analysis: the scanner always matches the longest possible lexeme to determine token boundaries.
• IN lexical-grammar-regular-syntactic-context-free
  Lexical grammars are regular languages (handled by FSMs with regular expressions); syntactic grammars are context-free languages (handled by parsers with stacks).
• IN lexing-two-stages-scanning-evaluation
  Lexing has two stages: scanning (segmentation into lexemes using a finite-state machine) and evaluation (converting lexemes into typed token values).
• IN lin-1998-similarity-formula
  Lin's (1998) similarity measure is sim(A,B) = 2·IC(LCS) / (IC(A) + IC(B)), normalizing by the information content of both compared concepts.
• OUT linear-cka-equals-rv-coefficient
  When computed on centered representations (using H = Iₙ − (1/n)11ᵀ), the RV coefficient exactly equals linear CKA: CKA_lin(X,Y) = ‖X_cᵀ Y_c‖²_F / (‖X_cᵀ X_c‖_F ‖Y_cᵀ Y_c‖_F)
• IN linear-model-orthogonal-symmetry
  In a linear model h=Wx, x'=W^Th+b, any orthogonal matrix O yields W'=OW with identical behavior because (OW)^T(OW) = W^TW, so no privileged basis exists.
• IN linear-projection-max-m-directions
  A purely linear projection from R^n to R^m can capture at most m independent directions, making the top-m principal components the optimal linear strategy.
• IN linear-representation-hypothesis-definition
  The Linear Representation Hypothesis states that neural network features are encoded as specific directions in activation space, with the map from features to activation vectors being linear even though the features themselves are nonlinear functions of the input.
• IN linear-transformers-equivalent-to-rnns
  Katharopoulos et al. (2020) showed that linear Transformers are equivalent to RNNs, and Schlag et al. (2021) proved they are 'secretly fast weight programmers'.
• IN liu-2023-20to50-diminishing-returns
  In open-domain QA, increasing retrieved documents from 20 to 50 yields only approximately 1-1.5% absolute accuracy improvement, indicating model reader saturation before retriever recall saturates.
• OUT liu-2023-claude-kv-near-perfect
  Claude-1.3 achieves near-perfect key-value retrieval across all tested lengths (140 and 300 key-value pairs), unlike other models that exhibit the U-shaped curve.
• IN liu-2023-contriever-retriever
  Distractor documents for the multi-document QA task are retrieved using Contriever (fine-tuned on MS-MARCO) from Wikipedia passages of 100 tokens or fewer.
• IN liu-2023-greedy-decoding-all-models
  All models in the lost-in-the-middle study use greedy decoding for generation.
• IN liu-2023-nq-open-queries
  The multi-document QA evaluation uses 2,655 queries from NaturalQuestions-Open with paragraph-type answers.
• OUT liu-2023-quadratic-self-attention
  Standard transformer self-attention requires memory and compute that grow quadratically (O(n^2)) with sequence length, historically limiting context windows to 512-2048 tokens.
• OUT llama-cpp-inference-framework
  llama.cpp is a C/C++ inference framework for local and cross-platform LLM deployment
• IN llama-exemplifies-craft-discipline-architecture-by-assembly
  Llama's architecture-by-assembly strategy — adopting proven refinements (SwiGLU, RoPE, RMSNorm) from independent prior research while focusing effort on data scaling — exemplifies the craft discipline's empirical epistemology: rather than innovating architecturally, Meta assembled the best empirically validated components and invested in the scaling lever the field had empirically identified as dominant.
• OUT llama-is-practical-open-weight-alternative
  Llama serves as a practical open-weight alternative to proprietary models, with competitive performance (13B beating GPT-3 175B) and local deployment infrastructure (llama.cpp).
• OUT llama-not-open-source-per-osi-fsf
  Llama is classified as source-available, not open-source, per OSI and FSF definitions; its acceptable use policy restricts military use (non-US), controlled substances, and critical infrastructure.
• OUT llama-training-data-scaling
  Llama training data scaled from 1.4T tokens (Llama 1) to 2T (Llama 2) to 15T (Llama 3) to 40T tokens (Llama 4 Scout).
• OUT llama-uses-swiglu-rope-rmsnorm
  Llama uses SwiGLU activation (not GeLU), rotary positional embeddings/RoPE (not absolute), and RMSNorm (not LayerNorm), differentiating it from GPT-3's architecture.
• OUT llama2-700m-dau-use-restriction
  Llama 2's license explicitly barred entities with more than 700 million daily active users from using the model, and prohibited using model outputs to train or improve other LLMs.
• IN llama2-70b-strategyqa-flip-direction-100pct-true-to-false
  On StrategyQA, 100% of observed answer flips for Llama2-70B are in the True→False direction.
• IN llama2-7b-confidence-over-95pct-both-settings
  Llama2-7B assigns >95% normalized token probability to its chosen answer in both single-source (counter-answer) and multi-source (memory-answer) settings.
• OUT llama2-rejection-sampling-before-ppo
  Llama 2 introduced rejection sampling before PPO in its RLHF fine-tuning pipeline, departing from the PPO-only approach.
• OUT llama3-8b-trained-15t-tokens-75x-chinchilla
  Llama 3 8B was trained on 15 trillion tokens, approximately 75x the Chinchilla-optimal 200 billion tokens, and performance continued to scale log-linearly.
• OUT llama4-maverick-codistilled-from-behemoth
  Llama 4 Maverick was codistilled from Behemoth (288B active / ~2T total parameters, 16 experts), while Scout was trained from scratch.
• OUT llama4-moe-scout-16-experts-maverick-128
  Llama 4 uses MoE architecture: Scout has 16 experts (17B active / 109B total parameters), Maverick has 128 experts (17B active / 400B total parameters).
• IN llamafile-single-executable-model
  llamafile bundles llama.cpp and model weights into a single executable file with optimized matrix multiplication kernels for x86 and ARM architectures.
• OUT lle-weight-constraints-ensure-rotation-translation-invariance
  LLE's reconstruction weight constraints (weights sum to 1 per row and are zero for non-neighbors) make the embedding invariant to rotation and translation of the input data
• IN llm-control-operates-at-training-and-inference-layers
  LLM behavior control has developed along two axes: training-time alignment (which diversified from RLHF into three independent paradigms — full RLHF, DPO/IPO/KTO, and Constitutional AI) and inference-time prompting (which evolved from linear CoT through self-consistency to branching ToT, adding search structure to manage prompt-dependent variability). The antecedents establish these as parallel developments but do not directly establish that they compensate for each other's specific limitations.
• OUT llm-energy-per-prompt-image-vs-text-60x
  Per Luccioni et al. (ACM FAccT 2024), a single image generation prompt requires ~2.91 Wh (up to 11.49 Wh for the least-efficient model) versus ~0.05 Wh for text generation/summarization, a roughly 60x energy difference
• OUT llm-evaluation-provides-stable-capability-assessment
  LLM evaluation through diverse benchmarking frameworks (MMLU, HLE, HELM, LMArena) provides stable, convergent capability assessments that reliably rank models and track progress.
• IN llm-field-is-fundamentally-craft-discipline
  The LLM field is fundamentally a craft discipline: both its most valuable structural properties (cross-boundary innovation, parameter redundancy) and its deepest barriers (tacit deployment knowledge, experiential prerequisites) are discovered and transmitted empirically, not through formal theory — meaning neither mastery nor failure modes are accessible through documentation alone.
• OUT llm-learning-involves-genuine-phase-transitions
  LLM capability acquisition involves genuine discontinuous phase transitions — both grokking (sudden generalization after memorization within training) and emergent abilities (capabilities appearing at scale thresholds) — rather than smooth, predictable improvement curves.
• IN llm-reliability-requires-five-independent-defense-dimensions
  LLM reliability requires independent defenses across at least five dimensions — two control layers (training-time alignment, inference-time prompting) and three security surfaces (training data poisoning, prompt injection, architectural vulnerability) — with no single defense sufficient on its own.
• OUT local-mlp-editing-principle
  The MLP-as-key-value-memory structure provides a principled, minimal-intervention editing mechanism for locally-stored factual (entity-relation-object) associations, with the edit's efficacy and specificity guaranteed by the rank-one update's geometric isolation in the covariance-projected key space.
• IN logit-difference-attribution-formula
  Logit-difference attribution ranks features by the quantity ∂(logit_A − logit_B) / ∂activation_i, measuring each feature's estimated causal contribution to the model's choice between two candidate next tokens.
• OUT long-tail-as-coordinate-coverage-gap-not-capacity-gap
  The long-tail knowledge failure is fundamentally a coordinate-coverage problem (the rank-one addressable subspace does not span rare fact directions) rather than a raw parameter-capacity problem, making the 10¹⁵-parameter scaling estimate a misleading reframing of a geometric coverage gap.
• OUT long-tail-as-geometric-addressability-failure
  The long-tail knowledge problem (Kandpal's 10¹⁵-parameter estimate, 176B-model failure on rare facts) is fundamentally a geometric addressability failure rather than a data-scarcity or model-capacity issue: tail facts fail to acquire well-conditioned individual directions in the residual stream because their key vectors lie in the poorly-conditioned tail of the covariance spectrum, making them inaccessible to both parametric recall (no clean MLP slot) and rank-one editing (C⁻¹k* becomes ill-conditioned), and correctly routed to the contextual channel instead.
• OUT lora-parallel-insertion-zero-inference-latency
  LoRA adds low-rank updates in parallel (computing Wx + BAx) rather than sequentially, enabling zero added inference latency because ΔW can be merged into W at serving time
• IN lord-ross-lepper-1979-polarization
  Lord, Ross & Lepper (1979) showed that after exposure to the same evidence about the death penalty, participants' attitudes became more extreme (polarization), demonstrating biased assimilation.
• IN lsa-non-incremental-vocabulary-limitation
  Latent Semantic Analysis requires a non-incremental vocabulary and has expensive pre-processing, limiting its use in dynamic or real-time settings.
• IN lsa-svd-term-document-matrix
  Latent Semantic Analysis (LSA; Landauer & Dumais, 1997) computes similarity via Singular Value Decomposition (SVD) of a term–document matrix, addressing synonymy and polysemy simultaneously.
• IN maml-finn-2017-optimizes-initialization-parameters
  MAML (Finn et al., 2017) is an optimization-based meta-learning method that trains initial parameters so that a small number of gradient steps on a new task yield good generalization.
• IN map-equal-priors-reduces-to-mle
  When all priors are equal (uniform), the Maximum A Posteriori (MAP) estimator reduces to the Maximum Likelihood Estimator (MLE).
• IN marginal-likelihood-integral-definition
  The marginal likelihood (evidence) is defined as p(X|α) = ∫ p(X|θ) p(θ|α) dθ, integrating out the parameter θ over the prior.
• IN market-amplification-targets-exactly-where-security-is-blind
  Market dynamics preferentially amplify the riskiest innovations — those crossing the most disciplinary and organizational boundaries — while security is reactive at precisely this unpredictable frontier, creating maximum market incentive for exactly the innovations that security cannot anticipate or prepare for.
• IN market-dynamics-preferentially-amplify-riskiest-innovations
  The adoption flywheel and persistent memory create a compound risk multiplier that preferentially targets the highest-value innovations — because boundary-crossing, paradigm-extending innovations are precisely what drives adoption acceleration — creating a natural selection mechanism where market dynamics systematically amplify the innovations carrying the greatest security debt.
• IN mata-v-avianca-6-fake-precedents-5000-fine-frcp-11
  In Mata v. Avianca (S.D.N.Y., 2023), the court dismissed the case and imposed a $5,000 fine for 6 fabricated precedents, citing Federal Rule of Civil Procedure 11 as the operative standard.
• IN mathematical-completeness-fails-to-guarantee-practical-reliability
  RLHF and prompting both illustrate cases where formal or mathematical specification proves insufficient for practical reliability: RLHF has a fully specified mathematical pipeline yet naive implementations fail without dozens of engineering details (motivating simpler alternatives like DPO/IPO/KTO), while prompting exhibits irreducible sensitivity and architectural injection vulnerabilities rooted in the model's inability to formally parse prompt structure. These two examples suggest that in at least some core LLM techniques, mathematical completeness or formal specification does not guarantee practical reliability.
• IN maximal-munch-rule-longest-match
  The maximal munch (longest match) rule requires the scanner to always match the longest possible lexeme from the input.
• IN mccarthy-proposed-circumscription-solution
  McCarthy proposed circumscription as a solution to the frame problem, a non-monotonic reasoning framework that minimizes the extension of abnormality predicates.
• OUT memit-extends-rome-multi-layer
  MEMIT extends ROME from a single-layer, single-edit closed-form update to a multi-layer, multi-edit bulk update applied simultaneously across all layers in the critical range R
• IN memorized-knowledge-diffuses-with-uncontrolled-weight-distribution
  Training data memorization as a dual-use property (knowledge source and extraction attack surface) becomes systematically more dangerous as model weights diffuse beyond governance capacity — because uncontrolled weight distribution makes memorized private data accessible to parties outside any licensing or governance framework.
• IN memory-security-surfaces-expand-across-temporal-levels
  The agentic memory architecture's three temporal levels — intra-turn consistency (ghost attention), inter-episode learning (reflexion), and cross-session persistence (dreaming) — likely introduce architecturally distinct security considerations that interact with the already-compounding security challenges from persistent state, suggesting a multi-dimensional attack surface where different temporal levels may require different defensive approaches.
• IN mend-hypernetwork-approach
  MEND trains a hypernetwork to produce weight updates from raw fine-tuning gradients for each edited fact
• IN meppelink-2019-health-literacy-confirmation-bias
  Meppelink et al. (2019) found that individuals with lower health literacy exhibit stronger confirmation bias when seeking online health information, especially in vaccination contexts.
• IN merrill-2022-saturated-transformers-threshold-circuits
  Merrill et al. (2022) showed that saturated (ReLU-activated) transformers are computationally equivalent to constant-depth threshold circuits, bounding their in-context computational power.
• IN meta-copyright-lawsuit-dismissed-june-2025
  Meta's copyright lawsuit by authors was dismissed by a U.S. court on June 25, 2025, in contrast to Anthropic's $1.5B settlement in the same period
• IN meta-learning-three-formal-requirements
  A meta-learning system must (1) include a learning subsystem, (2) gain experience from meta-knowledge extracted in previous episodes or across domains, and (3) choose its learning bias dynamically.
• IN mis-b-zero-iff-calibrated-not-monotonic-in-b
  With adaptive binning, Mis_b(g,p) = ‖p_{V_b(g)} − g‖_TV equals 0 if and only if g is calibrated to p for any b, and Mis_b is not monotonic in b: b=1 minimizes for g=uniform while b→∞ minimizes for g=p.
• IN mistral-alpha-copy-layers-15-17
  In Mistral 7B, the weekday index α is copied to the final output token at layers 15–17, explaining a drop in intervention effectiveness at those layers.
• IN mitigation-mechanisms-degrade-as-safety-deficit-intensifies
  The triple-bind safety deficit operates on a control stack whose both layers — training-time alignment and inference-time prompting — degrade structurally under the same agentic scaling that drives the market acceleration component of the deficit, creating a feedback loop where the deficit's primary mitigation mechanisms weaken precisely as the deficit intensifies.
• OUT mknn-null-baseline-scales-as-o-k-over-n
  The mutual k-NN (mKNN) neighborhood similarity metric has a null baseline of E[mKNN] = k/(n−1), scaling as O(k/n), which is far less confounded than spectral metrics because k ≪ d in practice and k is experimentally controllable.
• IN model-theoretic-formalization-paradox-free-minimal-models
  Model-theoretic formalization of non-monotonic logics restricts a monotonic logic's semantics to special models (e.g., minimal models) to derive sound and complete inference rules, eliminating paradoxes present in proof-theoretic (fixed-point) approaches; first-order circumscription, CWA, and autoepistemic logic were all successfully reformulated this way.
• IN moe-activates-subset-of-parameters
  Mixture of Experts (MoE) architecture activates only a subset of parameters per input token via a gating mechanism, reducing inference cost relative to dense models of equivalent total parameter count.
• IN moe-reduces-inference-cost
  Mixture of Experts (MoE) routes inputs to specialized sub-networks, reducing inference cost since only a fraction of parameters activate per input
• IN monofacts-estimator-error-bounds-within-o-1-sqrt-n
  The Good-Turing/MonoFacts estimator dMF = |{f : count(f) = 1 in x_train}| / n estimates the true missing mass p(U) within Õ(1/√n) with high probability, per McAllester and Schapire (2000).
• IN mt-and-speech-recognition-ai-complete
  Machine translation and speech recognition are classified as AI-complete problems, requiring comprehensive world knowledge
• IN mt-six-recognized-paradigms
  Machine translation is classified into six distinct paradigms: rule-based, statistical, example-based, transfer-based, neural, and computer-assisted.
• OUT mteb-aggregate-as-readout-bias
  MTEB's dataset-count-weighted aggregate score measures readout calibration breadth across task types rather than internal representation quality, because task-specificity is a readout-level phenomenon operating on top of a shared geometric substrate.
• OUT mteb-beir-15-retrieval-datasets
  MTEB's retrieval task includes 15 BEIR datasets: ArguAna, ClimateFEVER, CQADupstack, DBPedia, FEVER, FiQA2018, HotpotQA, MS-MARCO, NFCorpus, NQ, Quora, SCIDOCS, SciFact, Touche2020, TRECCOVID.
• OUT mteb-bitext-mining-metric
  The main metric for MTEB Bitext Mining is F1 score
• OUT mteb-eight-task-categories
  The eight MTEB task categories are: Bitext mining, Classification, Clustering, Pair Classification, Reranking, Retrieval, STS, and Summarization.
• OUT mteb-mteb-15-retrieval-datasets
  MTEB includes 15 publicly available retrieval datasets (e.g., MS-MARCO, NQ, FEVER, HotpotQA, ArguAna, ClimateFEVER) as part of its Retrieval task.
• OUT mteb-no-dominant-model-as-geometric-signature
  MTEB's observation that no single embedding model dominates all eight tasks is the expected geometric signature of a shared canonical semantic space probed by task-specific linear readouts, and the same second-moment structure that generates this task-specificity pattern is precisely what makes rank-one knowledge editing possible.
• OUT mteb-ranking-as-valid-comparator
  The MTEB leaderboard provides a valid comparative quality signal for embedding models, with the cosine→Spearman pipeline serving as a geometry-consistent evaluation...
• OUT mteb-scale-8-tasks-58-datasets-112-languages
  MTEB evaluates text embeddings across 8 task categories, 58 datasets, and 112 languages, benchmarking 33 models (open-source and API-based).
• OUT mteb-sickr-10000-sentence-pairs
  The SICK-R dataset in MTEB contains approximately 10,000 sentence pairs rich in lexical, syntactic, and semantic variation.
• OUT mteb-simultaneously-measures-read-and-edit-addressability
  MTEB's "no dominant model" observation is simultaneously a statement about readout diversity AND edit-addressability diversity: different embedding models optimize different directions in the same shared editable geometric space, making the benchmark a dual measure of read-quality and edit-coverage.
• OUT mteb-sts-score-range-0-5
  STS benchmark scoring in MTEB uses a 0–5 scale.
• OUT mteb-sts17-11-language-pairs
  STS17 cross-lingual STS benchmark covers 11 language pairs (Korean, Arabic, English, French, German, Turkish, Spanish, Italian, Dutch).
• OUT mteb-sts17-and-sts22-language-pairs
  STS17 covers 11 language pairs and STS22 expands to 18 language pairs in MTEB's cross-lingual semantic textual similarity evaluation.
• OUT mteb-sts17-sts22-language-pairs
  STS17 covers 11 language pairs and STS22 covers 18 language pairs for crosslingual evaluation.
• OUT mteb-sts22-18-language-pairs
  STS22 cross-lingual STS benchmark covers 18 language pairs, adding Polish, Russian, and Chinese to the STS17 set.
• OUT mteb-summarization-one-dataset
  MTEB summarization is represented by a single dataset (SummEval), making it the least-represented task and biasing the average score away from summarization
• OUT mteb-summarization-single-dataset-summeval
  Summarization is the most underrepresented MTEB task, represented by a single dataset (SummEval).
• OUT mteb-summeval-only-summarization
  SummEval is the only summarization dataset in MTEB.
• OUT mteb-task-categories-count
  MTEB defines exactly 8 task categories: Clustering, Classification, Pair Classification, Reranking, Retrieval, STS, Bitext Mining, and Summarization.
• OUT mteb-task-diversity-as-geometric-anisotropy
  MTEB's "no dominant model" result across 8 task types is the expected geometric signature of anisotropic readout in a shared semantic space: each task type (STS, retrieval, clustering, classification) probes a different projection direction, and task-specificity is the *predicted* outcome of geometric anisotropy, not a benchmark failure or model deficiency.
• OUT mteb-task-specificity-as-shrinkage-readout
  MTEB's "no dominant model" result is the readout-side signature of the same superposition that causes SAE shrinkage on the write side: the over-complete representation that prevents any finite SAE expansion ratio from fully decomposing the space also prevents any single embedding model from simultaneously optimizing all task-specific readout directions.
• OUT mteb-triple-structural-bias
  MTEB's aggregate score is triply biased—by dataset count weighting, by language restriction (retrieval/clustering English-only), and by task underrepresentation (summarization = 1 dataset)—making it a measure of readout-breadth coverage rather than of semantic quality
• OUT multi-hop-drop-as-rank-one-boundary-signature
  The dramatic 40.5%→7.4% multi-hop accuracy collapse after ROME editing is the operational fingerprint of the rank-one write boundary: multi-hop reasoning requires *composition* of multiple whitened directions, but the covariance-whitened subspace structurally supports only single-direction writes, making multi-hop parametric recall geometrically unreachable.
• OUT multi-model-editing-transferability
  Knowledge editing strategies (ROME-style local rank-one updates) are transferable across model families and architectures because the geometric structure of the residual stream is universal, provided that the geometric structure fully captures compositional (multi-hop) knowledge rather than only single-hop associations.
• OUT multimodal-attention-safely-generalizes-across-domains
  Attention's modality-agnostic universality and validated cross-modal deployment (ViT for images, AlphaFold for proteins) suggest that the Transformer safely generalizes as a universal computation substrate across new domains through tokenization alone.
• IN multimodal-fusion-early-vs-intermediate
  Two multimodal fusion strategies exist: early fusion (combine modality embeddings before prediction) and intermediate fusion (process modalities independently, then fuse via cross-attention, e.g., Flamingo).
• IN multimodal-transformers-validate-modality-independence
  The Transformer's successful deployment across text (GPT/BERT), images (ViT treating 16x16 patches as token sequences), protein sequences (AlphaFold), and game states (Decision Transformer) validates that the architecture's computational power is independent of input modality — any domain expressible as a token sequence can leverage attention's structured directed computation, with ViT's explicit patch-to-token mapping providing the clearest evidence of modality-agnosticism.
• IN mvu-embedding-as-semidefinite-programming
  Maximum Variance Unfolding (MVU) casts the low-dimensional embedding problem as a semidefinite programming problem, which carries high computational cost.
• IN myside-bias-rational-thinking-not-iq
  Myside bias correlates more strongly with rational-thinking ability (active open-mindedness) than with raw intelligence; highly intelligent individuals are not immune to confirmation bias.
• OUT mythos-discovered-271-firefox-150-vulnerabilities
  The restricted-access Mythos model discovered 271 zero-day vulnerabilities in Firefox 150 and Apple software flaws as part of Anthropic's Project Glasswing initiative to secure critical software using AI.
• OUT neighborhood-metric-null-baseline-o-k-over-n
  The expected mutual k-NN overlap under the null hypothesis is E_H₀[mKNN(X,Y)] = k/(n−1), scaling as O(k/n) with k ≪ d, making neighborhood metrics less susceptible to the width confounder than spectral metrics.
• IN ner-capitalization-unreliable-signal
  Capitalization is an unreliable signal for NER: German capitalizes all nouns, Chinese and Arabic have no capitalization, and sentence-initial words are always capitalized
• IN nli-model-validated-200-sample-annotation
  The NLI model used as an automated quality filter in the evidence-construction pipeline had its accuracy validated via 200-sample human annotation.
• OUT nlp-ai-complete
  Natural language processing (NLP) is classified as AI-complete, meaning solving it fully would require solving general artificial intelligence.
• IN nlp-ai-completeness-explains-craft-discipline-persistence
  NLP's classification as AI-complete — requiring human-level AI for general solutions — provides one theoretical explanation for why the LLM field operates as a craft discipline despite massive investment: if the underlying problem is inherently intractable for formal methods, this would help explain why empirical craft approaches persist, suggesting the craft-vs-formal gap may be a deep structural feature rather than merely a transitional state.
• IN nlp-cross-domain-cycle-from-importer-to-universal-substrate
  NLP's cross-domain evolution has come full circle: it imported foundational techniques from other fields (attention from machine translation, RLHF from robotics), synthesized them into the Transformer, then exported the result back as a universal computation primitive that now processes other fields' data through modality-agnostic tokenization — the importer became the universal substrate.
• IN nlp-evolution-is-empirically-driven-engineering-progression
  The NLP evolution from rules to agentic AI has been primarily an engineering-driven progression — the field's current pinnacle (autonomous agents operating code, GUIs, and design tools) was reached through accumulated engineering practices and efficiency innovations rather than theoretical breakthroughs, with standardized pipelines and reproducible practices compensating for gaps in theoretical understanding at successive stages of the stack.
• OUT nlp-pipeline-standard-ordering
  The standard NLP processing pipeline order is: tokenization → word segmentation → lemmatization/stemming → POS tagging → parsing → coreference resolution → semantic role labeling → sentiment analysis → NER → information extraction
• IN nlp-revolution-driven-by-cross-domain-technique-import
  The NLP paradigm shift from rules to attention architectures was significantly shaped by techniques that originated outside NLP — attention from machine translation augmentation and RLHF from game/robotics RL — rather than evolving solely from the rule-based tradition it replaced. This suggests that cross-domain technique transfer can serve as an important catalyst for field-level breakthroughs, though the evidence does not establish it as the sole or primary mechanism over NLP-native innovation.
• IN nlp-revolution-imported-techniques-then-exported-architecture
  The NLP revolution's cross-domain origins — attention imported from machine translation, RLHF from Atari/robotics — are mirrored by its cross-domain destination: the Transformer architecture exports back to protein folding, chess, and reinforcement learning, making NLP both a recipient and donor of foundational techniques across AI.
• OUT nlp-three-paradigms
  NLP has evolved through three major paradigms: symbolic/rule-based (1950s-1990s), statistical (1990s-2010s), and neural network methods (2010s-present)
• IN no-crosslingual-improvement-with-scale
  Increasing model size does not improve cross-lingual knowledge consistency, as measured by the RankC metric (Qi et al., 2023).
• IN non-monotonic-logic-addition-can-remove-conclusions
  In non-monotonic logic, adding a formula to the hypotheses can reduce (prune) the set of conclusions, in contrast to classical monotonic logics where adding premises never removes existing conclusions.
• IN non-privileged-basis-equivalence-lemma
  In a non-privileged basis (e.g., word embedding space), applying a random invertible linear matrix M to the embedding space and M⁻¹ to all downstream weights produces a functionally identical model with a different basis, demonstrating no direction is inherently special.
• IN nonlinearity-enables-overcomplete-encoding
  Introducing a single nonlinear activation (e.g., ReLU) in a 2-layer autoencoder breaks the rank-m constraint and enables representation of more features than bottleneck dimensions.
• IN now-metric-uses-dijkstra-on-wikipedia-dag
  The noW (n° of Wikipedia) metric constructs a directed acyclic graph from Wikipedia's hierarchy and applies Dijkstra's shortest-path algorithm to compute geodesic distance between topic nodes.
• IN ocs-formula-shen-2023
  Overlap Cosine Similarity (OCS) is computed as Σ_{t∈O} p₁(t)·p₂(t) / √[(Σ_{t∈O} p₁(t)²)(Σ_{t∈O} p₂(t)²)(K − |O|)] where O is the intersection of top-K token sets.
• IN open-weight-governance-failure-amplifies-irretirable-security-debt
  The open-weight ecosystem's governance failure is doubly compounding: definitional tensions mean there is no agreed standard for what "open" requires (Llama classified nonfree by FSF, OSAID demands training data disclosure), AND security debt becomes irretirable after weight release — meaning models enter the world under ambiguous governance frameworks that cannot address the permanent security implications of release, and no subsequent governance improvement can retroactively contain already-diffused weights.
• IN open-weight-models-face-unresolved-definitional-tensions
  The "open" AI ecosystem faces unresolved tensions: Llama's license restricts large platforms and prohibits competitive training use, the FSF classified it as nonfree software, and the OSAID requires training data disclosure that most "open" models do not provide.
• IN openai-copyright-ruling-nov-2024-evidence-controversy
  OpenAI received a temporary favorable ruling against news outlets' copyright suit on November 7, 2024, which was followed by a documented evidence-erasure controversy
• IN optimal-per-feature-l2-attack-formula
  The optimal per-feature L2 adversarial attack direction is attack_i = λ · (W^TW)_i / ||(W^TW)_i||_2, where λ is the perturbation budget and the feature i that maximally degrades model output is selected.
• IN opus-4-swe-bench-7-hour-continuous-coding
  Opus 4 set a SWE-Bench record by coding for 7 hours continuously in a single session (May 2025).
• IN original-paper-learned-positional-encodings-not-superior
  The original 'Attention Is All You Need' paper (2017) tested learned positional encodings and found them not superior to the sinusoidal positional encoding it ultimately used
• OUT osaid-could-resolve-open-weight-governance-gap
  The Open Source AI Definition (OSAID, October 2024) — requiring training data disclosure as a condition of the "open-source AI" label — could resolve the open-weight governance gap by establishing a clear, enforceable standard that retires the definitional tensions currently preventing coherent policy and enabling principled governance of model distribution.
• IN ot-baseline-preserves-local-not-global-structure
  The oracle-aided OT baseline achieves cos 0.70 but T⁻¹ = 0.00 and Rank ≈ 3064, preserving local similarity while failing to preserve global geometric rank ordering
• IN pagedattention-kv-cache-paging
  PagedAttention (used in vLLM) applies virtual memory-style paging to the KV cache, reducing GPU memory waste during inference.
• IN palm-google-540b-params-api-march-2023
  PaLM (Google) has 540 billion parameters and was opened via API in March 2023.
• IN paradigm-resilience-and-weight-diffusion-jointly-propagate-irremediable-vulnerability
  The pretrain-finetune paradigm's information-theoretically inseparable vulnerability propagation and weight diffusion's symmetric uncontainability of innovation and vulnerability operate as joint propagation mechanisms: the paradigm ensures vulnerability is baked into every model's capability (cannot be excised), while weight diffusion ensures every such model reaches actors who cannot manage its vulnerability (cannot be contained).
• IN paradigm-resilience-propagates-vulnerability-via-information-theoretic-identity
  The pretrain-finetune paradigm's propagation of vulnerability across the training pipeline is grounded in information-theoretic identity: capability and vulnerability are not merely co-located but informationally inseparable (compression quality equals memorization capability), meaning the paradigm's three-dimensional resilience (production validation, architectural survival, RLHF embedding) cannot be preserved while excising the vulnerability it carries.
• OUT parameter-redundancy-buffers-formally-ungrounded-agents
  Parameter redundancy — which provides empirical reliability despite insufficient formal understanding — may extend to buffer continuous agents as the apex of formally ungrounded engineering, with over-parameterized models absorbing perturbations that would break a tightly optimized system.
• OUT parametric-contextual-knowledge-duality
  LLMs maintain two functionally independent knowledge channels: parametric (editable via ROME's rank-one weight update) and contextual (supplied via retrieved passages that invert accuracy trends), which operate on separate mechanisms and can be independently manipulated.
• OUT parametric-sufficiency-for-head-facts
  Parametric MLP key-value storage is a sufficient mechanism for well-supported (head-of-distribution) knowledge, with the parametric/contextual duality providing a clean division of labor for knowledge storage and retrieval...
• OUT parametric-write-subspace-boundary
  The operational boundary between parametric recall and contextual retrieval is precisely the geometric boundary of the rank-one addressable subspace: facts whose subject-key projection aligns with the locally-stored key covariance are parametrically editable, while facts outside this subspace must be externally supplied.
• OUT park-2024-iclr-2025-venue
  Park et al. 2024 'Categorical and Hierarchical Concepts in S/R' was published at ICLR 2025.
• IN park2025-polytope-convex-hull
  A categorical concept {c₁, …, cₖ} maps to the convex hull (polytope) of the member vectors {v₁, …, vₖ} in representation space.
• OUT partial-whitening-cost-geometric-necessity
  The O(D²) cost of a rank-one ROME edit is the exact geometric price of single-direction partial whitening: modifying one row/column of the inverse covariance while preserving D−1 others is irreducibly O(D²) because C⁻¹ is a full D×D matrix, making the complexity bound a theorem of the geometry rather than an engineering limitation.
• IN pearson-knobloch-westwick-2019-online-vs-print
  Pearson & Knobloch-Westerwick (2019) found that the confirmation-bias gap in selective exposure is wider online than in print, particularly during pre-election periods.
• IN perceiver-latent-array-decouples-compute-from-input-length
  Perceiver and Perceiver IO use a small set of learned latent arrays to cross-attend to arbitrary-length inputs and then iteratively refine, decoupling computational cost from input sequence length.
• IN permutation-p-value-add-one-correction
  The permutation p-value is computed as p = (1 + #{null_scores ≥ s_obs}) / (K+1), where the +1 ensures p never equals zero and guarantees super-uniformity under exchangeability
• IN permutation-p-value-is-super-uniform
  The add-one permutation p-value p = (1 + #{s⁽ᵏ⁾ ≥ s_obs})/(K+1) is super-uniform under H₀, satisfying P(p ≤ α) ≤ α for all α ∈ [0,1], providing finite-sample Type-I error control without asymptotic distributional assumptions.
• IN perplexity-exponential-avg-neg-loglik
  Perplexity is the exponential of the average negative log-likelihood per token; lower perplexity indicates better prediction of a text corpus.
• IN persistent-memory-amplifies-compounding-security-challenges
  Persistent memory's transformation of agents from episodic to continuous amplifies the already-compounding security challenges across all maturity dimensions — turning session-scoped vulnerabilities into persistent attack surfaces while inheriting the craft-discipline barriers to developing adequate defenses.
• OUT persistent-memory-enables-long-horizon-autonomous-agents
  Persistent memory extending the agentic paradigm beyond session boundaries, combined with frontier agents' validated capability in high-stakes domains, enables a new class of long-horizon autonomous agents that accumulate operational expertise and pursue multi-session goals — a qualitative shift from single-session tool use to persistent autonomous operation.
• IN persistent-memory-extends-agentic-paradigm-beyond-context-windows
  Persistent memory (consolidating state between sessions, as in Claude's Dreaming feature) represents the next evolutionary step beyond context window expansion for the agentic paradigm — where context expansion provided the substrate for single-session agent capability, persistent memory enables cross-session continuity that is prerequisite for truly autonomous long-running agent workflows.
• IN persistent-memory-transforms-agentic-from-episodic-to-continuous
  Persistent memory (cross-session state consolidation) transforms the agentic paradigm — itself the culmination of the entire NLP evolution — from episodic tool use bounded by context windows into continuous autonomous operation with temporal coherence, enabling agents to pursue long-horizon goals across sessions rather than single-task episodes.
• IN phase-change-finite-system-not-thermodynamic
  In the Elhage 2022 paper, 'phase change' refers to a discontinuous transition in a finite system, explicitly not a thermodynamic phase transition requiring infinite system-size limits.
• IN positional-encoding-shift-diagonal-matrix-property
  The sinusoidal positional encoding has the property that position shifts are expressible as diagonal matrix multiplications: f(t + Δt) = diag(f(Δt)) · f(t), enabling relative position learning as linear transformations and convolutions as linear operations.
• IN practical-agentic-ai-is-culmination-of-entire-nlp-evolution
  The agentic AI paradigm — LLMs autonomously operating code, GUIs, and design tools — is the culmination of the entire NLP evolution: the decoder-only paradigm shift made practical by the efficiency stack created capable base models, while context window expansion (enabled by those same efficiency techniques) provided the substrate for multi-step autonomous operation.
• IN pre-ln-eliminates-warmup-requirement
  Pre-LN (layer normalization before attention/FFN sublayers) stabilizes training and eliminates the need for learning rate warmup, unlike the original post-LN design from 2017.
• IN pretrain-finetune-is-most-resilient-nlp-contribution
  The pretrain-finetune paradigm demonstrates unique resilience across three independent dimensions — production validation at web scale (every Google Search query), survival as methodology across architectural transitions (BERT's encoder-only design gave way to decoder-only dominance), and embedding in the RLHF reward model as a living instance — making it arguably the most durable single methodological contribution in NLP history.
• OUT pretrain-finetune-paradigm
  The Transformer training pipeline follows a pretrain-finetune paradigm: self-supervised pretraining on large unlabeled corpora, then supervised fine-tuning on task-specific data, with optional alignment via RLHF or Constitutional AI.
• IN pretrain-finetune-resilience-exemplifies-craft-discipline-mechanism
  The pretrain-finetune paradigm's resilience across three dimensions (production validation, architectural survival, methodological embedding in alignment) provides strong evidence that the craft discipline can produce durable engineering patterns — this resilience emerged through empirical deployment validation rather than theoretical proof, illustrating a primary epistemic mechanism characteristic of the craft discipline.
• IN pretrain-finetune-resilience-propagates-vulnerability-across-pipeline
  The pretrain-finetune paradigm's demonstrated resilience across three dimensions (production validation, architectural survival, and RLHF embedding) coexists with memorization's dual-use characteristics (contributing to model knowledge while creating extraction and poisoning attack surfaces). Because the paradigm persists and propagates across the model pipeline, any memorization-related vulnerabilities present within it may persist as well — though this connection is an inference from the co-occurrence of these two independently established properties rather than a directly demonstrated causal relationship.
• IN privileged-basis-created-by-activation-functions
  A privileged basis in neural networks is created by architecture-induced symmetry breaking, specifically by applying a per-neuron activation function (e.g., ReLU), which makes individual neuron directions special and encourages features to align with them.
• IN procrustes-3d-rotation-via-svd-kabsch
  In 3D Procrustes analysis, the optimal rotation is a 3×3 rotation matrix found via Singular Value Decomposition of the cross-covariance matrix, where the constraint det(R) = 1 (no reflection) is known as the Kabsch algorithm
• IN procrustes-distance-is-sqrt-of-ssd
  The Procrustes distance is defined as the square root of the sum of squared distances (SSD) between corresponding landmark points after optimal superimposition, not the SSD itself
• IN prompt-automation-paradox-exemplifies-systematic-theory-gap
  Prompt engineering's paradox — automatable (LLMs generate prompts at human-level quality) yet fundamentally fragile (model-specific, 40%+ accuracy shifts from formatting) — is a microcosm of the field's systematic pattern where engineering capability outpaces theoretical understanding: the field can build tools that generate effective prompts without understanding why they work, mirroring its broader ability to deploy what it cannot formally specify.
• IN prompt-fragility-amplified-across-all-memory-timescales
  Prompt fragility amplifies across all memory timescales: context expansion widens the intra-session attack surface (more tokens create more injection opportunities and sensitivity triggers), while persistent memory extends that vulnerability across sessions — the dual architectural limitation of prompt control (sensitivity and injection) compounds along both axes of the memory evolution from wider windows to persistent state.
• IN prompt-fragility-compounds-with-context-expansion
  The dual architectural limitation of prompt control — sensitivity (40%+ accuracy shifts from formatting) and injection (inability to distinguish instructions from input) — compounds with the 10,000x context window expansion enabled by efficiency breakthroughs: larger contexts create proportionally larger attack surfaces for injection attempts, make sensitivity effects harder to diagnose across expanded input, and increase the probability that adversarial content co-occurs with legitimate instructions in the same context window.
• IN prompt-theory-gap-makes-inference-control-structurally-unimprovable
  The inference-time control layer's structural self-undermining (prompt fragility compounding with context expansion while authority hierarchies create attack surfaces) may be particularly resistant to incremental improvement: the systematic theory gap exemplified by prompt engineering's paradox (automatable yet fundamentally fragile, with engineering capability outpacing formal understanding) suggests that at least some of the inference layer's vulnerabilities reflect structural properties resistant to formalization rather than straightforward engineering deficits awaiting better solutions.
• IN promptsource-ide-for-natural-language-prompts
  PromptSource (ACL 2022) is an IDE/repository for version-controlling and sharing natural-language prompts, analogous to GitHub for code.
• IN prototypical-networks-classify-by-prototype-distance
  Prototypical Networks classify by computing the distance to class prototypes in an embedding space.
• IN python-offside-rule-context-sensitive
  Python's off-side rule requires the lexer to maintain a stack of indent levels and emit INDENT/DEDENT tokens, making the lexical grammar context-sensitive.
• IN python-offside-rule-indent-dedent
  Python's off-side rule is implemented at the lexer level: the lexer maintains a stack of indent levels and emits INDENT/DEDENT tokens, making the lexical grammar context-sensitive rather than context-free.
• IN quadratic-attention-spawned-comprehensive-efficiency-stack
  Self-attention's O(n²) complexity is a primary driver of efficiency research, motivating both sub-quadratic algorithmic alternatives (sparse/linear variants, Performers) and a multi-layer practical optimization stack spanning algorithmic (speculative decoding), memory (PagedAttention), and compute (FlashAttention) approaches.
• IN quadruple-intractability-compounded-by-invisible-structural-flaw
  The quadruply intractable safety deficit (untargetable, unscalable, market-accelerated, irreversible) operates on an alignment layer whose most dangerous structural flaw — the capacity inversion between pretraining and alignment — is permanently invisible to the practitioners who would need to address it, meaning the deficit is not only intractable along four axes but actively concealed from would-be responders.
• OUT rag-reliable-for-knowledge-augmentation
  RAG provides reliable knowledge augmentation — reducing hallucinations and keeping responses current via external retrieval at the prompting layer, with practical value scaling alongside context window expansion that enables richer retrieval payloads.
• OUT raw-bert-sts-worse-than-glove
  Out-of-the-box BERT embeddings (mean or CLS) perform worse than average GloVe embeddings on STS tasks before fine-tuning.
• OUT re2c-2-3x-faster-than-flex
  re2c produces lexers 2-3x faster than flex by generating directly-coded lexers (goto statements) instead of table-driven lexers.
• IN re2c-faster-than-flex
  The lexer generator re2c (directly-coded approach) produces engines 2-3x faster than flex (table-driven approach).
• IN re2c-generates-faster-lexers-than-flex
  re2c generates directly coded state transitions for lexers and is reported to be 2–3× faster than flex's table-driven finite-state machine approach
• OUT reactive-security-addressable-through-dual-layer-defense-maturation
  The security deficit at the unpredictable innovation frontier could become addressable as both the training-time alignment layer (diversified into three independent paradigms) and inference-time control layer (increasingly sophisticated prompting from CoT through ToT) independently mature — each layer catches failure modes the other misses, and their independence means a single-point failure cannot compromise both.
• OUT read-write-asymmetry
  A structural asymmetry governs LLM knowledge: the read channel (feature-level interpretation via SAE, Park polytope analysis, embedding evaluation) operates in a rich, model-independent geometric space, while the write channel (editing) is impoverished to parametric rank-one MLP updates in a single layer—superposition enriches the feature space for reading but makes feature-level writing intractable.
• OUT read-write-asymmetry-superposition-consequence
  The read-broad/write-narrow asymmetry is the direct operational consequence of over-complete superposition: reading is a simultaneous linear projection over all active features, but writing must be rank-one to avoid cross-talk in the over-complete basis, making the write channel inherently and necessarily narrower.
• OUT read-write-geometric-asymmetry-topology
  The LLM is architecturally a read-broad/write-narrow system: the read channel (SAE feature extraction, evaluation, retrieval) spans the full d-dimensional covariance-whitened space, while the write channel (ROME rank-one, SAE feature ablation) is restricted to a 1-dimensional key-direction subspace, making knowledge correction fundamentally more constrained than knowledge interrogation.
• IN reader-saturation-20-documents
  In open-domain QA, language-model reader accuracy plateaus at approximately 20 retrieved documents; beyond that, additional documents yield only ~1.5% (GPT-3.5-Turbo) or ~1% (Claude-1.3) marginal accuracy gains while retriever recall continues to improve.
• OUT readout-head-as-geometric-projection
  The 48-point MEAN-vs-CLS pooling gap (77.03 vs 29.19) on STSb is the cleanest single experimental proof that task-specificity is a readout projection effect: the identical superposed residual stream yields radically different geometric subspaces depending solely on which linear projection (readout head) is applied.
• OUT reasoning-advances-reflect-genuine-discontinuity
  Both training-time reasoning specialization (o1 scoring 6x better than GPT-4o on IMO problems, R1 matching proprietary models at open-weight cost) and inference-time structured reasoning evolution (CoT → self-consistency → tree-of-thoughts) reflect genuine cognitive discontinuities rather than smooth scaling artifacts.
• IN reasoning-and-alignment-are-orthogonal-capability-dimensions
  Reasoning capability and alignment quality appear to be separable dimensions in LLM development — reasoning can be independently addressed at both training time (o1, R1) and inference time (CoT, ToT), while alignment has diversified into three independent paradigms (RLHF, DPO/KTO, Constitutional AI) — suggesting that the capability-alignment design space may be at least a 2×2 matrix (training/inference × capability/alignment), with each quadrant potentially offering independent optimization levers.
• OUT reasoning-models-are-genuine-cognitive-discontinuity
  Reasoning-specialized models demonstrate a genuine cognitive discontinuity — with o1 scoring 83% vs GPT-4o's 13% on IMO problems, consistent with the broader observation that emergent abilities appear discontinuously at scale thresholds — unless the apparent discontinuity is an artifact of metric choice rather than a real capability transition.
• IN reasoning-orthogonality-creates-combinatorial-configuration-space
  The orthogonality of reasoning and alignment as capability dimensions, combined with LLM control operating at independent training and inference layers, creates a combinatorial model configuration space — reasoning can be tuned independently (training-time specialization or inference-time prompting), alignment can be selected from three paradigms, and both operate across two control layers — replacing a single capability-safety tradeoff with a multi-axis design space.
• IN reflexion-agent-episodic-memory
  The Reflexion agent method uses episodic 'lessons learned' as long-term memory across multiple episodes
• IN reft-modifies-hidden-activations-not-weights-on-frozen-model
  ReFT (Representation Fine-Tuning) modifies less than 1% of hidden representations (activations) rather than weights on a fully frozen base model, distinguishing it from weight-based PEFT methods like LoRA.
• IN regex-cannot-handle-recursive-nesting
  Regular expressions cannot handle recursive or nested patterns (e.g., matching nested parentheses); that requires a parser with a stack.
• IN reliable-deployment-requires-irreducible-operational-expertise
  LLM deployment reliability is shaped by two compounding challenges: defense-in-depth strategies that are both necessitated and bounded by theoretical gaps in formal guarantees, and a persistent accessibility gap driven by divergent optimization requirements between training and deployment — together suggesting that empirical operational expertise remains a critical bottleneck that tooling and documentation alone are unlikely to fully address.
• IN relu-hidden-layer-model-equation
  The privileged-basis ReLU hidden layer model is defined as h = ReLU(Wx), x' = ReLU(W^Th + b), where the ReLU activation breaks rotational symmetry and makes W directly interpretable.
• IN reptile-nichol-2018-first-order-approximation-maml
  Reptile (Nichol et al., 2018) is a first-order approximation of MAML that avoids second-order derivatives, making it cheaper but less accurate.
• OUT residual-stream-universal-substrate
  SAE (middle-layer residual stream), ROME (mid-layer MLP value projection), and Park (final-layer unembedding) all identify the residual stream at different depths as the primary locus of interpretable geometric structure.
• IN resnik-1995-similarity-formula
  Resnik's (1995) similarity measure is defined as sim(A,B) = IC(LCS(A,B)), using only the information content of the lowest common subsumer.
• IN retrieval-augmentation-constrained-by-inference-control-limits
  Retrieval augmentation's evolution from flat to structured knowledge (RAG to GraphRAG), having recapitulated the broader NLP evolution at the context layer, is constrained by the same structural limits that make inference-time control unimprovable: the systematic theory gap in prompt engineering means retrieval quality is subject to the same prompt fragility, injection vulnerability, and model-specific sensitivity that limits all prompting-layer approaches.
• IN retrieval-evolution-recapitulates-nlp-evolution-at-context-layer
  Retrieval augmentation's evolution from flat to structured knowledge (RAG to GraphRAG) recapitulates the broader NLP pattern of progressing from unstructured to structured representations — but operating at the retrieval/prompting layer rather than the model layer. This parallel suggests that the field's engineering-driven evolutionary patterns may recur at different architectural levels, though whether this constitutes true scale-invariance remains an open question.
• OUT retrieval-inversion-as-geometric-signature
  The inversion of accuracy trends—parametric accuracy *decreases* with document count while contextual accuracy *increases*—is the operational signature of the geometric addressability boundary: the inversion occurs precisely at the subspace boundary (where the fact direction exits the rank-one addressable space), not at a frequency threshold, making the boundary a geometric rather than statistical object.
• OUT reward-model-alignment-resilient-against-base-poisoning
  The RLHF reward model — grounded in the surviving pretrain-finetune paradigm's three-dimensional resilience and backed by a complete mathematical pipeline (Bradley-Terry, KL penalty, PPO) — provides resilient alignment signals that reliably encode human preferences.
• IN reward-model-inherits-vulnerability-from-paradigm-it-evaluates
  The RLHF reward model — itself an instance of the surviving pretrain-finetune paradigm — may inherit the paradigm's vulnerability propagation characteristics: memorization as a dual-use property could flow from pretraining through the reward model into alignment scoring decisions, potentially creating a circular dependency where the judge inherits the defendant's flaws. However, this connection is inferred from the co-occurrence of paradigm resilience and memorization's dual-use nature rather than directly demonstrated.
• IN rg65-gold-standard-dataset-65-pairs
  RG65 is a gold-standard evaluation dataset containing 65 human-judged word pairs used to benchmark computational semantic similarity measures.
• OUT riesz-isomorphism-unifies-embedding-and-unembedding
  The Riesz isomorphism (Park et al. 2023, Theorem 3.2) shows the map ϕ(γ̄) = ⟨γ̄, ·⟩_C sends each unembedding representation γ̄_W to the corresponding embedding representation λ̄_W, unifying measurement and intervention via the causal inner product.
• OUT riesz-map-as-canonical-semantic-isomorphism
  The Riesz map under the causal (whitened) inner product is the unique canonical isomorphism that makes embedding and unembedding semantically identical, thereby elevating "meaning" from a model-specific activation pattern to a well-defined algebraic object in a model-independent vector space
• OUT riesz-map-as-key-value-semantic-bridge
  The Riesz isomorphism under the causal inner product is the mathematical "compiler" that translates between ROME's key representation (MLP input activation) and value representation (MLP output projection), unifying the parametric knowledge storage mechanism as a single dual-geometric object rather than two independent weight matrices.
• OUT riesz-map-as-unified-operational-framework
  The Riesz map under the causal inner product is the unique canonical object that simultaneously defines the evaluation coordinate system (cosine/Spearman as inner-product measurement), the editing mechanism (ROME's key-value bridge via pre-/post-activation isomorphism), and the semantic equivalence of embedding and unembedding—making all three operations different projections of a single canonical structure.
• OUT riesz-map-unifies-full-logical-structure
  The Riesz map under the causal inner product is the unique mathematical object that simultaneously proves cross-model convergence is necessary (not contingent), enables the polytope/orthogonal space decomposition, and provides the canonical embedding↔unembedding isomorphism—collapsing three independent observations into theorems of a single structural premise (over-complete superposition in a shared substrate).
• IN rlhf-completeness-paradox-motivated-simpler-alternatives
  RLHF has a fully specified mathematical pipeline (Bradley-Terry, KL penalty, PPO clipping), yet DPO, IPO, and KTO independently eliminated its separate reward model and RL loop by collapsing alignment into a single optimization step with shared β-controlled KL regularization — suggesting that the multi-component pipeline's complexity was a significant factor motivating simpler alternatives.
• OUT rlhf-fragility-drove-parallel-simplification-and-sophistication
  RLHF's completeness paradox (fully specified mathematics, practically fragile without dozens of engineering details) drove the field in two simultaneous directions: simplification (DPO/IPO/KTO eliminating the RL loop entirely) and sophistication (Llama 2's rejection sampling, PPO-ptx pretraining gradients) — the field responded to fragility by both reducing and mastering complexity in parallel.
• IN rlhf-irreducible-complexity-validates-alignment-diversification
  RLHF's irreducible theoretical complexity — optimal policies are inherently non-Markovian, and online and offline formulations diverge fundamentally — independently validates the field's diversification into simpler alignment alternatives (DPO, KTO, Constitutional AI): the complexity is a theoretical ceiling, not merely an engineering inconvenience, making alternatives necessary rather than just convenient.
• OUT rlhf-is-reproducible-engineering-discipline
  RLHF is a reproducible engineering discipline with a complete mathematical specification (Bradley-Terry loss, KL penalty, PPO clipping) and proven production outcomes (InstructGPT, ChatGPT).
• OUT rlhf-k-wise-more-efficient-than-pairwise
  Using K-wise comparisons directly in RLHF is asymptotically more efficient than converting them into pairwise comparisons
• OUT rlhf-pipeline-has-structural-robustness-asymmetry
  The RLHF pipeline exhibits a structural robustness asymmetry: its reward model inherits the battle-tested pretrain-finetune paradigm's stability (the same methodology that survived decoder-only's displacement of encoder-only), while its RL optimization layer remains brittle enough to motivate elimination via DPO/IPO/KTO.
• OUT rlhf-preference-evaluation-reliable-at-scale
  RLHF preference evaluation scales reliably: the foundational "easy to judge" property is amplified by K-wise ranking efficiency (asymptotically more efficient than decomposing into pairwise comparisons), enabling robust preference data collection at scale.
• OUT rlhf-reward-model-cross-entropy-bradley-terry
  The RLHF reward model is trained with a cross-entropy loss over pairwise preferences using the Bradley-Terry-Luce (BTL) probabilistic model: L(θ) = -1/C(K,2) * E[log σ(r_θ(x, y_w) - r_θ(x, y_l))].
• OUT rlhf-reward-model-scaling-vs-data
  In RLHF, scaling up reward model size is more effective than scaling up data volume for improving performance
• IN rlhf-shared-ancestry-creates-reward-model-evaluation-blindspot
  RLHF's architecture — two models initialized from the same pretrained ancestor — means the reward model inherits the same memorization patterns, biases, and knowledge gaps as the policy it evaluates, creating a systematic evaluation blindspot where the judge cannot detect vulnerabilities it shares with the defendant.
• OUT rlhf-three-stage-pipeline-is-standard-alignment
  RLHF's three-stage pipeline (SFT → reward model → PPO) remains the standard proven alignment method, having produced both InstructGPT and ChatGPT.
• OUT rlhf-trains-reward-model
  RLHF works by training a reward model on human preference comparisons, then fine-tuning the LLM via reinforcement learning to maximize that reward
• OUT rome-edit-as-partial-whitening
  A rank-one ROME weight update is operationally equivalent to a local, single-direction whitening of the residual stream: the C⁻¹k* projection in the update formula performs precisely the covariance-normalization that superposition necessitates, but confined to one key direction—making each edit a "partial whitening" that corrects one superposed feature without disturbing the orthogonal complement.
• OUT rome-gpt2xl-intervention-layer-18
  ROME intervenes at layer 18 in GPT-2 XL, identified as the center of causal effect in MLP layers where MLP outputs shift from key to value behavior.
• IN rope-dot-products-relative-position-only
  RoPE (Rotary Positional Embedding) encodes position by rotating pairs of embedding dimensions; its key property is that dot products depend only on relative position, not absolute position.
• IN rsa-exception-to-width-bias
  RSA (Spearman rank correlation of dissimilarity matrices) is an exception to width-driven bias: as a self-normalized correlation of dissimilarities, its null stays near zero with no width-driven drift even without calibration
• IN rte-three-outcomes-entailment-contradiction-independence
  Recognizing Textual Entailment (RTE) has exactly three possible outcomes for a pair of text fragments: entailment, contradiction (entailment of the negation), or independence.
• IN rubenstein-goodenough-1965-similarity-relatedness-distinction
  Rubenstein & Goodenough (1965) established the foundational distinction between human-rated synonymy (similarity) and relatedness, forming the basis for all subsequent evaluation datasets.
• OUT rwkv-rnn-transformer-alternative
  RWKV (Peng et al. 2023) is an alternative architecture that combines RNN efficiency (linear complexity in sequence length) with transformer-level performance
• IN s-sparse-assumption-exponential-gap-facts-hallucinations
  The s-sparse facts assumption states |F| ≤ e^(−s)·|H| with probability 1 over D_world, encoding that there are exponentially more plausible false factoids than true ones (e.g., paper citations, 5W combinations).
• OUT sae-feature-causal-reliability
  SAE-identified features serve as causally meaningful interpretability primitives, with ablation producing predictable logit-space effects that correlate strongly with downstream behavior.
• OUT sae-feature-id-format
  SAE feature IDs in the Templeton 2024 paper follow the pattern /, e.g., 1M/570621 (unsafe code), 1M/1013764 (code error), 34M/1385669 (backdoor).
• OUT sae-functional-abstraction-extends-geometric-scope
  SAE features activating on functional analogies (transit feature on wormholes) and cross-modal inputs (text-trained features firing on images) demonstrate the geometric space encodes intensional and relational structure beyond Park's extensional categorical polytopes, broadening the geometric framework's explanatory scope to include non-lexical, compositional semantics.
• OUT sae-granularity-as-superposition-resolution
  SAE's empirical granularity scaling (broad categorical features at small run sizes → specific entity features at large run sizes) is the operational resolution of superposition: the 10–200× expansion ratio creates discrete "zoom levels" at which the same underlying covariance geometry manifests as features of different semantic specificity.
• OUT sae-guided-feature-space-editing
  SAE-identified features provide a semantically-interpretable coordinate system for specifying knowledge edits—enabling edits to be expressed as feature-space operations (e.g., "suppress feature 34M/31164353 and amplify its neighborhood") that the covariance geometry guarantees map to valid rank-one weight-space operations—thereby bridging the interpretability and editing literatures through the shared second-moment structure.
• OUT sae-neighborhood-as-directsum-navigation
  The SAE feature neighborhood (Golden Gate Bridge → Alcatraz → San Francisco → California) is not merely a navigation path within a single categorical polytope but the operational traversal algorithm for the full direct-sum decomposition of the semantic space: it simultaneously navigates the categorical subspace (discrete concepts) and the hierarchical orthogonal subspace (WordNet parent-child structure), validated across Gemma-2B and LLaMA-3-8B.
• OUT sae-neighborhood-as-polytope-navigation
  SAE feature neighborhoods (e.g., Golden Gate Bridge → Alcatraz → San Francisco → California) are the operational navigation algorithm for the categorical polytope geometry: each SAE feature is a polytope vertex, the neighborhood structure is the polytope edge adjacency, and cross-model universality confirms this polytope is a shared semantic object rather than a model-specific artifact.
• OUT sae-resolution-of-space-decomposition
  SAE's expansion-ratio scaling (broad→specific features) is the operational resolution mechanism for the algebraic space decomposition: coarse SAEs resolve the polytope hulls (categorical structure), fine SAEs resolve individual vertices (entity-level features), and the neighborhood adjacency graph is the polytope edge structure visible at whichever resolution is chosen.
• OUT sae-shrinkage-as-finite-resolution-limit
  The SAE shrinkage problem (under-reconstruction with finite expansion ratios) is the operational signature of finite resolution in the superposition→whitening framework: any finite dictionary size leaves irreducible reconstruction loss because the superposed structure is fundamentally over-complete, and the power-law decrease in loss with compute is the scaling signature of approaching (but never reaching) full resolution.
• IN safety-and-expertise-crises-unified-by-craft-epistemics
  The LLM field's safety deficit and expertise crisis appear to be connected manifestations of the field's empirical epistemic mode: the craft discipline's experiential knowledge-building mechanism is both the source of its most resilient contributions (validated through deployment rather than design) and a primary driver of the safety deficit's intractability (the same experiential requirement that builds expertise makes it non-transferable and unable to scale with adoption demand, while market dynamics preferentially amplify the highest-risk innovations).
• IN safety-crisis-accelerated-by-multimodal-attack-surface-expansion
  The LLM safety crisis's convergent runaway dynamics (self-reinforcing craft-epistemic loop compounded by degrading mitigation) are accelerated by attention's modality-agnostic universality coupling capability and vulnerability expansion — each new modality (text, image, audio, protein) multiplicatively extends both the system's value and its attack surface, with security expertise unable to keep pace across domains.
• IN safety-crisis-exhibits-convergent-runaway-dynamics
  The LLM safety crisis exhibits convergent runaway dynamics: the self-reinforcing craft-epistemic loop (where the methodology generating safety deficits also blocks their detection) operates simultaneously with the structural degradation of both control layers (training-time alignment and inference-time prompting), meaning the crisis accelerates while the mechanisms that could counteract it deteriorate under the same forces.
• IN safety-crisis-irreversible-and-self-concealing
  The LLM safety crisis is both irreversible AND self-concealing: the convergent runaway dynamics and doubly permanent security debt make reversal structurally impossible, while the entrenched expertise crisis — compounded by innovation diffusion spreading challenges to unprepared institutions and invisible capacity inversion hiding structural flaws — ensures practitioners lack the knowledge to recognize the dynamics they are embedded in.
• IN safety-crisis-irreversible-because-mitigation-degrades-while-debt-accumulates
  The LLM safety crisis is irreversible under current structural conditions: the convergent runaway dynamics (self-reinforcing craft-epistemic loop compounding with degrading control layers) operate on a foundation of monotonically accumulating, doubly permanent security debt (informationally unpatchable and distributionally irretirable), meaning even if the runaway dynamics were somehow arrested, the historical debt cannot be unwound.
• IN safety-crisis-maximally-intractable-invisible-and-surface-expanding
  The LLM safety crisis is simultaneously maximally intractable along all known dimensions (quadruply so — untargetable, unscalable, market-accelerated, irreversible — compounded by the invisible structural flaw of capacity inversion hidden beneath the most intractable deployment challenges), AND its attack surface is actively expanding through multimodal universality that structurally couples every new capability domain with new vulnerability surfaces — creating a crisis that cannot be diagnosed (invisible), cannot be addressed (intractable), and is growing faster than even a hypothetical response could track (surface-expanding).
• IN safety-crisis-theoretically-inescapable-and-practically-irreversible
  The LLM safety crisis admits no resolution pathway under current structural conditions: it is practically irreversible (convergent runaway dynamics where the methodology generating the deficit also degrades the mechanisms for addressing it, compounded by doubly permanent security debt) AND theoretically inescapable (NLP's AI-completeness makes formal verification impossible, so the craft methodology that generates the crisis cannot be transcended by formal methods — there is no alternative epistemology to escape to).
• IN safety-deficit-compounded-by-irretirable-historical-security-debt
  The doubly intractable safety deficit is permanently compounded by historical security debt: every past weight release embeds irretirable vulnerabilities (training data memorization diffusing through uncontrolled distribution), meaning even a revolutionary future safety methodology would face a monotonically growing backlog of deployed, unaddressable attack surfaces.
• IN safety-deficit-doubly-intractable-untargetable-and-unscalable
  The LLM safety deficit is doubly intractable: it is untargetable because the frontier's next capability surprise cannot be predicted, AND unscalable because the expertise paradox ensures that even addressing known safety gaps requires experiential knowledge that cannot be mass-produced — two independent mechanisms of persistence that make the deficit self-reinforcing regardless of resource allocation.
• IN safety-deficit-intractable-in-both-dimensions-and-response
  The LLM safety deficit is intractable along every axis: the deficit itself is doubly intractable (untargetable because the frontier is unpredictable, unscalable because expertise resists formalization), and the field's best available response (alignment diversification) was theoretically inevitable yet practically insufficient — the challenge and the response to it are both structurally inadequate.
• IN safety-deficit-is-structural-and-widening
  The LLM field faces a structural and widening safety deficit: safety assurance is fundamentally informal (craft-based, not formally verifiable) while the security expertise gap widens with each capability advance, so the field's safety capacity cannot keep pace with growing demand.
• IN safety-deficit-quadruply-intractable-and-irreversible
  The LLM safety deficit is quadruply intractable and irreversible: untargetable (the frontier's next capability surprise cannot be predicted), unscalable (craft expertise cannot grow with adoption), market-accelerated (dynamics preferentially amplify the riskiest innovations), AND irreversible (every historical weight release embeds permanent, irretirable security debt that accumulates monotonically).
• IN safety-deficit-triple-bind-untargetable-unscalable-and-market-accelerated
  The LLM safety deficit is triply intractable: it is untargetable (the frontier's next capability surprise cannot be predicted), unscalable (expertise accumulation is inherently experiential and cannot keep pace with adoption demand), AND market-accelerated (the adoption flywheel preferentially amplifies the highest-risk innovations because boundary-crossing capabilities carry the most security debt and attract the most investment).
• IN safety-deficit-untargetable-due-to-unpredictable-frontier
  The structural safety deficit is not merely widening but fundamentally untargetable: because the field cannot predict its own next capability surprises, safety investment cannot be directed at specific future threats — the deficit grows in a direction that cannot be anticipated, making proactive safety assurance impossible in principle.
• OUT safety-formalization-could-follow-pretrain-finetune-trajectory
  The craft discipline's fundamentally informal safety assurance could formalize through the same mechanism that made pretrain-finetune resilient — repeated production validation eventually crystallizing tacit safety knowledge into standardized methodology, following the paradigm's proven three-dimensional resilience pattern.
• OUT safety-investment-monotonically-improves-user-experience
  Higher safety classification and Constitutional AI alignment principles produce monotonically improving model behavior — safety investment in tiered capability management and principle-based alignment translates directly into better, more reliable user interactions across the capability spectrum.
• OUT sbert-triplet-loss-euclidean-epsilon-1
  SBERT's triplet loss uses Euclidean distance (not cosine) with margin ε = 1.
• IN scaling-evidence-is-itself-empirical-validating-craft-methodology
  Key scaling relationships in LLM research — such as power-law relationships between performance and resources (Kaplan et al., 2020) and Chinchilla's information-theoretic grounding of compute-optimal scaling — were discovered through empirical observation rather than first-principles derivation. That these foundational quantitative regularities emerged from empirical methods is consistent with the field's broader character as a craft discipline where core knowledge is discovered and transmitted experientially.
• IN scaling-laws-generalize-across-training-paradigms
  Chinchilla's information-theoretic grounding of scaling in compression quality, combined with the observation that all three training task types (masked, autoregressive, prefix) are compression variants, implies that scaling law relationships should generalize across training paradigms — not just the autoregressive models where they were empirically measured.
• OUT scaling-laws-kaplan-2020-power-law
  Kaplan et al. (2020, arXiv:2001.08361) established power-law relationships between model performance and compute, data, and parameters
• IN schmidhuber-1987-thesis-foundational-meta-learning
  Schmidhuber's 1987 thesis 'Evolutionary principles in self-referential learning' is the foundational reference for the meta-learning field.
• OUT scoped-factual-knowledge-lifecycle
  For single-hop factual (entity-relation-object) knowledge specifically, the full lifecycle—distributed corpus acquisition → local MLP key-value storage → parametric recall → contextual supplementation → rank-one correction—forms a coherent, validated closed loop in which each stage is geometrically well-defined and operationally distinct.
• IN security-and-expertise-locked-in-mutual-dependency
  The LLM field's security and expertise challenges form a vicious cycle: security mitigation requires deployment experience that creates the exposure, while the expertise needed to deploy safely cannot scale because craft knowledge resists formalization — each constraint reinforces the other, preventing the system from reaching equilibrium.
• IN security-challenge-compounds-across-all-maturity-dimensions
  LLM security is uniquely difficult because challenges compound across every dimension of the field's maturity: attack surfaces expand with capability scaling, defense-in-depth strategies are simultaneously necessitated and bounded by theoretical gaps, and the empirical nature of the field means security expertise — like all operational expertise — is irreducibly experiential rather than formally transferable.
• IN security-debt-doubly-permanent-informationally-and-distributionally
  The LLM security debt is doubly permanent along independent dimensions: informationally unpatchable (capability and vulnerability are the same compressed information — removing memorized data degrades capability) AND distributionally irretirable (weights already released cannot be recalled and embed the vulnerability permanently), making the training-data security surface not merely difficult but impossible to remediate through any known mechanism.
• IN security-debt-irretirable-after-weight-release
  The security debt independently generated by both technical advances and market dynamics becomes significantly harder to retire once model weights are released: weight diffusion makes the training-data security surface fundamentally uncontainable after release, meaning that accumulated security debt from memorized training data and poisoned inputs propagates irreversibly through the ecosystem, though other security surfaces (such as prompt injection) may remain partially addressable through post-release mitigations.
• IN security-expertise-gap-widens-with-each-capability-advance
  Each LLM capability advance simultaneously creates new dual-use security surfaces that scale superlinearly and requires new operational expertise to defend them — expertise that can only be acquired through direct experience with the new capability's failure modes — creating a security expertise gap that widens with every generation.
• IN security-is-reactive-at-unpredictable-frontier
  LLM security is doubly reactive at the frontier: the field cannot predict which capabilities will emerge next, and the frontier where those capabilities appear is precisely where security understanding is weakest — security teams are always preparing for the last surprise, not the next one.
• IN security-mitigation-requires-the-deployment-that-creates-the-exposure
  The compounded security exposure from adoption acceleration and deployment divergence creates a vicious cycle with the recursive experiential barrier: mitigating security exposure requires deployment expertise that can only be acquired through the very deployments that create the exposure, with no external entry point to break the cycle.
• OUT security-vulnerability-detection-scales-safely-with-capability
  AI-powered security analysis scales safely with model capability — frontier models find hundreds of real vulnerabilities (271 in Firefox) while multi-agent collaboration demonstrates production-grade code generation (C compiler in Rust), suggesting security-capable AI is a net defensive asset.
• OUT self-attention-permutation-invariant-without-positional-encoding
  Self-attention is permutation-invariant without positional encodings, meaning token order has no effect on output unless positional information is explicitly injected.
• IN semantic-similarity-biomedical-go-functional-not-sequence
  In biomedical informatics, semantic similarity compares gene/protein function via GO terms and is distinct from and complementary to sequence similarity (e.g., BLAST); tools include ProteInOn, CMPSim, and CESSM.
• IN semantic-similarity-restricted-to-hypernymic-relations
  Semantic similarity measures only 'is-a' (hypernymic) relations between terms, whereas semantic relatedness is broader and includes antonymy, meronymy, and any associative link.
• IN semicolon-insertion-and-line-continuation-complementary
  Semicolon insertion (Go, JavaScript) adds a token absent from source based on newlines and contextual cues, while line continuation (Bash, Python) suppresses a newline token that would otherwise be generated — the two mechanisms are complementary
• OUT sentence-bert-siamese-architecture-snli-training
  Sentence-BERT (Reimers & Gurevych, 2019) uses a Siamese BERT network architecture fine-tuned on the SNLI dataset (entailment/contradiction/neutral triplet labels) to produce sentence embeddings.
• IN senteval-conneau-kiela-2018
  The SentEval toolkit (Conneau & Kiela, 2018) is a standardized multi-task benchmark suite for evaluating universal sentence representations across STS, classification, regression, and paraphrase tasks.
• IN separability-index-correlated-irreducibility
  A higher separability index S(f) (minimum mutual information over all rotations) indicates a feature is more separable and thus more likely irreducible.
• IN separation-logic-authors-ohearn-reynolds-yang
  Separation logic was developed by O'Hearn, Reynolds, and Yang for reasoning about shared mutable data structures.
• IN separation-logic-frame-rule-star-connective
  In separation logic, the frame rule uses the * connective ('and separately') for disjoint memory regions, allowing arbitrary unmodified memory to be added to pre/post specifications without altering the code's specification.
• IN sft-one-epoch-confirms-inverted-scaling-dynamics
  The InstructGPT paper's deliberate restriction of SFT training to a single epoch — explicitly to avoid overfitting — independently confirms the inverted scaling dynamics between pretraining and alignment: pretraining thrives on data volume (15T+ tokens in Llama 3) while alignment degrades from the same abundance, requiring intentional information restriction at the pipeline transition point.
• IN sglang-multimodal-inference
  SGLang is an inference engine that supports both LLMs and multimodal models
• IN sglang-supports-multimodal-model-inference
  SGLang is an inference engine that explicitly supports both text LLMs and multimodal models, with a focus on structured generation, distinguishing it from text-only serving frameworks
• IN sgpt-bias-only-tuning
  SGPT uses fewer than 0.1% of pre-trained parameters for contrastive fine-tuning (bias-only tuning)
• IN shanahan-1997-solving-frame-problem-mit-press
  Shanahan's 1997 book 'Solving the Frame Problem' (MIT Press) is the definitive mathematical treatment of the frame problem.
• IN shuffling-contradiction-resolution
  The apparent contradiction between the 2025 empirical control (shuffling unembeddings *destroys* the full orthogonality structure) and the set-inclusion finding (shuffled unembeddings *reproduce* the child-parent⊥parent sub-orthogonality) is resolved by scope: the *full* four-condition Theorem 8 structure is genuine and shuffle-destroyed, while a *specific* sub-condition (consecutive-level parent⊥child-parent) is a set-inclusion artifact—meaning the geometric framework captures the genuine full structure, not the spurious sub-structure.
• IN siamese-network-shared-weights-paired-embedding
  A Siamese network architecture uses two identical sub-networks with shared weights to process paired inputs, training a loss function that enforces closeness for similar pairs and separation for dissimilar pairs in embedding space
• IN siamese-networks-share-identical-weights
  Siamese Networks share identical weights between their two branches and learn a similarity/distance function over sample pairs.
• IN sick-r-pair-count
  SICK-R contains approximately 10,000 sentence pairs that are lexically, syntactically, and semantically rich
• OUT single-fact-edit-efficacy
  ROME's high efficacy and specificity on individual factual associations holds as a reliable operational claim, but is scoped to single-hop entity-relation edits.
• IN snli-corpus-574k-sentences
  The SNLI corpus (Bowman et al., 2015) contains 574k sentences and serves as training data for NLI-based sentence encoders like InferSent.
• OUT space-decomposition-under-superposition
  The full LLM semantic space admits a clean algebraic decomposition into categorical polytope subspaces (discrete concepts) and hierarchical orthogonality subspaces (graded taxonomic structure) within the covariance-geometric framework, providing a complete account of how discrete and graded meaning coexist in a single over-complete vector space
• OUT speculative-decoding-draft-model-parallel-verify
  Speculative decoding uses a smaller draft model to generate candidate tokens, then the larger model verifies them in a single parallel forward pass, exploiting that verification is cheaper than sequential generation.
• IN speculative-decoding-draft-model-verify
  Speculative decoding uses a smaller draft model to propose tokens that a larger model then verifies, accelerating inference by generating multiple tokens per forward pass of the large model.
• IN speculative-decoding-inference-optimization
  Speculative decoding and PagedAttention are key inference optimization techniques for LLMs
• IN stanovich-2009-iq-vs-rationality
  Stanovich (2009) argues that standard IQ tests measure certain analytical abilities but miss broader dimensions of rational thought such as recognizing cognitive biases and probabilistic reasoning.
• IN stanovich-intelligence-not-protective-confirmation-bias
  Stanovich's vehicle-ban experiment found no IQ difference in myside bias, indicating that raw intelligence does not protect against confirmation bias; rational-thinking disposition and active open-mindedness are the protective factors.
• IN stargate-llc-is-openai-joint-venture-entity
  Stargate LLC is a joint venture entity associated with OpenAI, presumably for data-center or infrastructure investment, not a model or product.
• IN static-vs-contextual-embedding-assignment
  Static embeddings (word2vec, GloVe, fastText) assign one fixed vector per word regardless of context, while contextual embeddings (ELMo, BERT) produce a unique vector per token occurrence, with the static approach inherently unable to distinguish polysemous senses
• OUT structured-reasoning-overcomes-factual-accuracy-gap
  The evolution of structured reasoning prompting (CoT → self-consistency → ToT) combined with training-time reasoning specialization provides systematic methods to close the factual accuracy gap between LLMs and humans.
• OUT sts17-language-pairs
  STS17 in MTEB covers 11 crosslingual language pairs
• IN successor-state-axiom-per-fluent-pattern
  A successor state axiom has the form: for each fluent F, F(t+1) ↔ (action makes F true) ∨ (F(t) ∧ action does not make F false).
• OUT superposition-as-compositional-basis
  Superposition is the fundamental compositional mechanism in LLMs: the 10–200× over-complete expansion (SAE), the key-value memory structure (ROME's W_fc/W_proj), and the direct-sum space decomposition (Park's polytope+orthogonality) are three independent geometric consequences of the same over-completeness.
• OUT superposition-as-single-root-cause
  Superposition is the unique root cause from which the full read/write/editing logical structure follows: over-complete representation necessitates covariance whitening (making geometry well-defined), which in turn explains both the broad-read/narrow-write asymmetry (projection vs. rank-one injection) and the closed triangle linking all three properties—there is no independent second principle needed.
• OUT superposition-covariance-editability-triangle
  Superposition, covariance whitening, and rank-one editability form a closed logical triangle in which each property necessitates the others: over-complete superposition requires covariance separation for feature addressability, covariance separation defines the geometric space in which rank-one updates are well-defined, and the boundedness of rank-one editing confirms the addressable space is finite.
• OUT superposition-geometry-explains-universality
  The cross-model universality of feature geometry (SAE features more similar across architectures than within, Park orthogonality validated on both Gemma and LLaMA) is a consequence of superposition: the over-complete compositional basis is determined by the shared semantic grammar of language, making geometric structure an architectural invariant rather than a model-specific artifact.
• OUT superposition-necessitates-covariance-whitening
  Over-complete superposition in the shared residual-stream substrate is the precise structural condition that necessitates covariance/whitening (second-moment projection) as the canonical tool for isolating individual features and performing targeted rank-one edits; without superposition, raw Euclidean geometry would suffice and the entire covariance-geometry framework would be unnecessary.
• OUT superposition-unifies-data-and-representation-redundancy
  Over-complete superposition is the single structural cause of both the data-side redundancy (5× correlated corpora yield only marginal gains, Spearman 0.87–0.97 inter-correlation) and the representation-side redundancy (SAE dead features at 2–48%, reconstruction shrinkage): in both cases the available "capacity" exceeds the unique information it must encode.
• IN symbolic-ai-models-do-not-hallucinate
  Symbolic AI models generally do not produce hallucinations; the phenomenon is specific to generative and statistical models like LLMs.
• OUT task-specificity-emerges-from-readout
  Task-specificity in embedding quality is a readout phenomenon: the internal feature geometry is largely model-independent (convergent across architectures), while MTEB's no-dominant-model result arises because each task's unembedding/projection head selects a different subspace of the same shared geometric structure.
• OUT task-specificity-vs-feature-universality
  Embedding model quality is irreducibly task-specific (no single model dominates MTEB's 8 tasks) while internal feature representations are largely model-independent (SAE features transfer across architectures), separating output-space competition from internal geometric convergence.
• IN technical-and-market-mechanisms-independently-generate-security-debt
  The LLM field's security debt is generated by two independent and mutually reinforcing mechanisms — technical (attention efficiency enabling capabilities whose security implications were never designed for) and market (adoption dynamics preferentially amplifying the riskiest innovations) — making the total security challenge multi-causal and resistant to any single-vector solution.
• IN ten-instruction-phrasings-stability-screened
  Ten instruction phrasings were tested in preliminary runs, and only the most output-stable formulation was carried into the final experiments.
• IN tetlock-fox-hedgehog-two-decade-study
  Tetlock's fox/hedgehog study tracked political pundits over two decades and found that 'foxes' (multiple-hypothesis thinkers) were significantly more accurate than 'hedgehogs' (dogmatic single-theory thinkers).
• IN text-to-image-word-order-matters-no-negation
  In text-to-image models, word order matters (earlier words are weighted more heavily), negation is often not understood, and these models do not process grammar the same way LLMs do
• IN textual-entailment-three-outcomes
  Textual entailment has three possible outcomes: entails, contradicts, or neutral — not just true/false
• IN textual-inversion-learn-new-words-from-3-5-images
  Textual Inversion (Gal et al., 2023) personalizes a frozen text-to-image model by learning new embedding-space 'words' from as few as 3–5 images without retraining model weights.
• IN textual-inversion-learned-word-embedding-from-images
  Textual inversion (Gal et al. 2023) optimizes a new word-embedding from a set of example images and inserts the resulting 'pseudo-word' token into prompts to condition generation in frozen text-to-image models.
• IN theorem-1-capacity-exponential-bounds
  Theorem 1 establishes that the number of pairwise δ-orthogonal projection matrices in a d-dimensional residual stream has a lower bound of e^{C₁(d/d′²)δ²} and an upper bound of e^{C₂(d−d′)δ² log(1/δ)}, showing exponential capacity with a large gap between bounds.
• IN three-classic-action-representation-problems
  The three classic problems of action representation are the frame problem (unchanged facts), the qualification problem (implicit preconditions), and the ramification problem (implicit consequences).
• IN three-discrete-feature-learning-regimes
  Neural network feature learning exhibits three discrete regimes (not learned, superposition, dedicated orthogonal dimension) with sharp discontinuous transitions between them.
• OUT three-knowledge-conflict-types-taxonomy
  The Xu et al. (EMNLP 2024) survey defines three knowledge conflict types: context–memory conflict, inter-context conflict, and intra-memory conflict.
• OUT three-level-memory-enables-expertise-codification
  Agentic memory spanning three temporal levels — combined with the field's craft knowledge being fundamentally experiential — could enable systematic codification of deployment expertise across sessions, potentially addressing the expertise scalability paradox by distributing experiential knowledge through persistent agent memory rather than requiring each practitioner to acquire it independently.
• IN three-phase-resolution-framework
  The survey proposes a three-phase conflict-resolution framework: (1) training phase to mitigate internal parameter inconsistency, (2) generation phase with algorithmic decoding intervention, (3) post-hoc phase with post-processing correction.
• OUT three-tier-architecture-completeness
  The three-tier memory architecture (broad superposition read, narrow parametric write, unbounded context write) constitutes a complete and faithful model of LLM knowledge management
• OUT three-tier-memory-architecture
  The LLM implements a three-tier memory architecture: broad superposition-based read, narrow rank-one parametric write confined to the covariance-whitened addressable subspace, and unbounded contextual write via the context window—with the geometric boundary of C⁻¹k* defining the precise demarcation between parametric and contextual regimes.
• OUT three-transformer-variants-encoder-decoder
  Three main transformer architectural variants exist: encoder-only (BERT) for understanding, decoder-only (GPT) for autoregressive generation, and encoder-decoder (T5, original transformer) for sequence-to-sequence tasks.
• IN token-segmentation-2-n-1-possible-splits
  A string of length |s| has 2^(|s|−1) possible token segmentations, illustrating the combinatorial explosion inherent in token boundary identification.
• IN tokenization-is-critical-unsettled-compression-bottleneck
  Since compression is the fundamental scaling variable and the input representation layer (tokenization strategy and positional encoding) remains actively contested, tokenization may be among the most impactful yet least settled components of the scaling pipeline — the compression process begins at the input layer where raw data is first reduced, making this contested design space potentially significant for scaling outcomes.
• IN tower-of-babel-paradox-local-encoding-privacy
  The Tower of Babel Paradox (Brette, 2017) states that if each neuron's local encoding is maximally efficient (indistinguishable from random to neighboring neurons), then no global encoding or synchronization is possible across a large neural network, creating a theoretical obstruction to biologically plausible learning
• IN training-and-deployment-optimization-diverge-at-every-level
  LLM training and deployment require fundamentally divergent optimization strategies: training prioritizes data volume over parameters (validated by both Chinchilla theory and compression evidence), while deployment requires a comprehensive efficiency stack to manage quadratic attention costs — meaning optimal LLM development demands different expertise and infrastructure at each lifecycle stage.
• IN training-data-security-surface-permanently-permeable-after-release
  Training data memorization diffusing through uncontrolled weight distribution makes the training-data security surface — one of three independent surfaces requiring defense — fundamentally uncontainable after model release, as once weights are distributed the memorized knowledge and any poisoned training data are irreversibly in the wild.
• IN training-deployment-divergence-amplifies-security-surfaces
  The divergence between training and deployment optimization can amplify security challenges: training-time priorities (e.g., massive data volume for quality) may create deployment-time attack surfaces, while deployment-time defenses operate under different constraints than those that shaped training — and since security challenges compound across all maturity dimensions of the field, addressing vulnerabilities introduced at one lifecycle stage from another stage is inherently difficult rather than straightforward.
• OUT transformer-architecture-paper-vaswani-2017-neurips
  The foundational transformer architecture paper is Vaswani et al., 'Attention Is All You Need,' presented at NeurIPS 2017.
• OUT transformer-attention-is-all-you-need-2017
  The transformer architecture was introduced in the 'Attention Is All You Need' paper by Google in 2017.
• OUT transformer-attention-remains-necessary-for-frontier-quality
  Transformer self-attention remains necessary for frontier-quality language modeling, with no alternative architecture achieving comparable results at scale — the 2017 architecture's core mechanism is still load-bearing.
• OUT transformer-chess-static-evaluation-no-search-elo2895
  A transformer using only static chess board evaluation (no Minimax or alpha-beta search) achieved an Elo rating of 2895, equivalent to grandmaster level
• OUT transformer-introduced-2017-attention-paper
  The Transformer architecture was introduced in the paper 'Attention Is All You Need' by Vaswani et al., published at NeurIPS 2017.
• OUT transformer-introduced-2017-neurips
  The transformer architecture was introduced in 2017 at NeurIPS in the paper 'Attention Is All You Need' by Google researchers.
• IN transformer-paper-vaswani-2017
  The transformer architecture was introduced by Vaswani et al. in 2017 in the paper 'Attention Is All You Need' presented at NeurIPS
• IN triplet-loss-from-facenet
  Sentence-BERT's triplet loss formulation (minimize distance(anchor, positive) − distance(anchor, negative) + margin) is borrowed from FaceNet (Schroff et al., 2015).
• IN truthfulqa-contains-817-questions
  TruthfulQA is a benchmark consisting of exactly 817 questions designed to mimic common falsehoods and idiom-based traps that LLMs tend to replicate
• IN tsne-2d-projection-loses-higher-dim-structure
  t-SNE visualization projects latent/embedding space to 2D for human inspection but does not faithfully preserve all higher-dimensional relationships, making 2D plots unreliable for interpreting true embedding geometry
• IN umap-adds-repulsive-term-over-tsne
  UMAP differs from t-SNE by adding a repulsive term for far points, which improves preservation of global structure, whereas t-SNE focuses primarily on local neighborhood preservation
• IN uncertainty-ratios-six-evidence-configurations
  Uncertainty ratios are measured across six evidence configurations: 0/2, 1/3, 1/2, 2/4, 2/3, and 2/2 (supportive vs. contradictory evidence counts).
• OUT uncontainable-innovation-diffusion-serves-collective-progress
  The institutional uncontainability of LLM innovation — craft knowledge crossing organizational and disciplinary boundaries, with breakthrough techniques escaping their originators — serves collective capability progress by ensuring no single institution bottlenecks frontier advancement.
• IN universal-dependencies-cross-linguistic-standard
  Universal Dependencies is the cross-linguistic standard for syntactic annotation
• OUT validated-frontier-agents-generalize-to-adversarial-settings
  Frontier agents validated in cooperative high-stakes domains (16 Opus 4.6 agents writing a C compiler, Mozilla patching 271 Firefox vulnerabilities with Mythos), combined with the adoption flywheel's momentum toward agentic deployment, suggest that agentic capabilities are ready to generalize from cooperative settings to adversarial deployment environments where external actors may attempt to manipulate agent behavior.
• IN varibad-zintgraf-2019-variational-autoencoder
  VariBAD (Zintgraf et al., 2019) is a model-based meta-reinforcement learning method that uses a variational autoencoder to capture task information in internal memory and conditions the policy on that task.
• OUT vec2vec-compute-176-gpu-days-55-models
  The vec2vec experimental matrix required approximately 176 GPU days of training (25 full models × 4 days + 30 partial models × 2 days + 1 Qwen pair × 20 days), 42 GPU hours of evaluation, and 45 CPU hours for OT baselines, using NVIDIA 2080Ti → L4 → A40 → A100 GPUs.
• IN vibe-coding-collins-word-of-year-2025
  'Vibe coding' was named Collins Dictionary Word of the Year in November 2025, referring to an AI-assisted software development workflow driven by iterative conversational prompting.
• IN von-oswald-2023-dispute-phase-shift
  von Oswald et al. (2023) disputed Akyürek et al.'s phase-shift claim, arguing that larger Transformers still implement gradient descent but with data transformations.
• IN von-oswald-2023-training-hyperparameters
  The von Oswald et al. (ICML 2023) experiments use Adam optimizer (lr=0.001 for K<3, 0.0005 for K≥3), batch size 2048, gradient clipping (global norm=10), Haiku fan-in truncated normal init (std=0.002/K), no regularization, token clipping to [−10,10] for K>2, W₀=0 initialization, and the Optax library.
• IN walters-v-openai-may-2025-ruled-for-openai
  In Walters v. OpenAI (May 2025), the court ruled in OpenAI's favor because the plaintiff failed to prove negligence or actual malice regarding a hallucinated legal summary.
• IN wason-coined-confirmation-bias-1960
  Peter Wason introduced the term 'confirmation bias' in 1960 following his rule-discovery experiments in which participants were given the example (2,4,6) and the true rule was 'any ascending sequence.'
• IN wason-selection-task-1968
  In the Wason Selection Task (1968), participants overwhelmingly select confirming cards rather than falsifying ones, demonstrating confirmation bias in rule-testing.
• IN weight-availability-catalyzes-ecosystem-regardless-of-intent
  Model weight availability — whether deliberate (BERT open-sourced November 2018) or accidental (Llama 1 leaked via BitTorrent March 2023) — acted as a catalyst for wider adoption in both cases, suggesting that weight access may be an important factor for driving community engagement and downstream activity.
• IN weight-availability-outpaces-governance-capacity
  The open-weight ecosystem exhibits a structural governance gap: weight availability catalyzes adoption regardless of licensing intent (BERT open-sourced, Llama leaked via BitTorrent), while the definitional tensions around "openness" (OSI/FSF disagreements, restrictive acceptable use policies, training data disclosure requirements) remain unresolved — meaning the ecosystem grows faster than governance frameworks can constrain it.
• IN weight-availability-outpaces-governance-capacity-v2
  The open-weight ecosystem shows an emerging tension between adoption dynamics and governance: weight availability — whether deliberate (BERT open-sourced, 2018) or accidental (Llama 1 leaked via BitTorrent, 2023) — appears to act as a catalyst for community engagement and downstream activity irrespective of licensing intent, while definitional tensions around 'openness' remain unresolved (Llama's license restricting large platforms and competitive training use, the FSF's classification as nonfree software, and disclosure requirements that most 'open' models do not meet). Together, these suggest that adoption may be proceeding ahead of the resolution of governance and definitional questions, though the antecedents do not establish a persistent structural gap or an unresolvable speed asymmetry.
• IN weight-diffusion-makes-innovation-and-vulnerability-symmetrically-uncontainable
  Weight diffusion creates parallel uncontainability dynamics for both innovation value and security vulnerabilities, though through distinct mechanisms operating in related domains: craft discipline knowledge transfer makes foundational contributions institutionally uncontainable (as seen in Google's Transformer/BERT becoming universal infrastructure), while weight release makes the training-data security surface permanently permeable once weights are distributed — suggesting that the openness enabling cross-boundary progress in craft disciplines also propagates security vulnerabilities from training data memorization, though the symmetry is structural rather than mechanistic.
• IN weight-tying-input-output-embeddings
  Weight tying (Press & Wolf, 2017) uses the same matrix for both input and output embeddings in language models, improving performance; it is a standard technique in modern Transformers.
• IN width-null-energy-random-matrix-theory
  Under H₀ (independence), the expected squared Frobenius norm of the sample cross-covariance is E[‖Ĉ‖²_F] = dₓd_y/(n−1), so when dₓ and d_y are comparable to n, the null energy is O(n) and does not vanish as n→∞
• IN wolfe-britt-metacognitive-argument-belief
  Wolfe & Britt found that personal belief is not the source of myside bias; rather, a person's metacognitive belief about what constitutes a good argument (e.g., 'a good argument is fact-based') predicts the bias, and balanced-research instructions mitigate the effect.
• IN wordnet-structure-synsets-hypernym-meronym
  WordNet is structured with synsets, hypernym/hyponym links, and meronym (part-of) relations, and serves as the canonical lexical resource for English semantic similarity benchmarks.
• IN xlnet-permutation-masking-generalizes-causal
  XLNet generalizes standard causal masking by applying a random permutation P: P · M_causal · P⁻¹, allowing different tokens to have different masking patterns rather than a single fixed left-to-right order.
• IN xu-2024-cantor-pairing-flattens-2d-enumeration
  Xu et al. (2024) use the Cantor pairing function k = (i+j)(i+j+1)/2 + j to flatten the 2D enumeration of (model index i × training-state index j) into a single sequence {ĥ₀, ĥ₁, …}, enabling the diagonalization construction of a defeating ground-truth function.
• OUT xu-2024-hallucination-formal-impossibility-proof
  Xu, Jain & Kankanhalli (NUS, 2024, arXiv:2401.11817v2) formally proved that hallucination is ineliminable for all computable LLMs because no computable LLM can learn all computable functions, reclassifying hallucination from an engineering bug to an intrinsic architectural limitation
• IN xu-2024-inevitability-theorem-core
  For all computable LLMs h, there exists a computable linear ordering < on finite strings such that h hallucinates on the question 's_{2n+1} < s_{2n}?' regardless of how many training samples are provided (Xu et al. 2024).
• IN xu-2024-polynomial-time-llms-belong-to-ce-set
  All polynomial-time-bounded LLMs (i.e., all current LLMs) belong to a computably enumerable set, placing them within the scope of Theorems 1 and 2 of Xu et al. (2024) without requiring the stronger Theorem 3.
• IN xu-2024-survey-github-repo
  The Xu et al. 2024 knowledge conflicts survey is available at https://github.com/pillowsofwind/Knowledge-Conflicts-Survey.
• IN xu2024-context-memory-87pct-drop
  Misinformation in context causes up to 87% performance drop in ChatGPT on the NQ-1500 and CovidNews datasets (Pan et al. 2023, as reported in Xu et al. 2024 Table 2).
• IN xu2024-intra-memory-contradiction-rate
  Intra-memory contradictory output probability ranges from 15.7% to 22.9% across models (Mündler et al. 2023), with stronger models producing fewer contradictions.
• IN xu2024-multi-turn-belief-alteration-range
  Multi-turn belief alteration in LLMs ranges from 20.7% to 78.2% cumulative shift as counter-memory rounds increase (Xu et al. 2023, as reported in the survey).
• IN xu2024-survey-search-methodology
  The Xu et al. 2024 survey restricted its literature search to publications after 2021 using Google Scholar and ACL Anthology with key terms "knowledge conflict," "knowledge inconsistency," and "knowledge gap," expanded via forward and backward citation snowballing.
• IN yale-shooting-problem-counterexample-circumscription
  The Yale shooting problem is a counterexample demonstrating that circumscription-based minimization can produce incorrect inferences in certain temporal scenarios.
• IN yang-tsay-chan-2002-longest-match-segmentation
  Yang, Tsay, and Chan (2002) formally analyzed the applicability and limitations of the longest-match rule for dictionary-based word segmentation, published in Computer Languages, Systems & Structures, Vol. 28(3), pp. 273–288.