core-llm-techniques-transferred-from-outside-nlp-v2

IN premise

Created 2026-08-24T18:41:00+00:00

Two significant LLM techniques — attention (which evolved from an RNN-augmentation mechanism in NMT to a standalone architecture) and RLHF (which transferred from Atari/robotics to NLP, enabled by a domain-independent property: tasks that are hard to specify but easy to judge) — each reached prominence through a notable methodological shift. RLHF's case involves a clear cross-domain transfer from non-NLP settings into NLP; attention's involves an architectural reconfiguration within NLP's own trajectory. Together, these two cases suggest that both technique evolution and cross-domain transfer have each contributed to important LLM advances, though two examples alone do not establish their frequency or relative importance compared to NLP-native innovation.

Summary

This notes that two of the most consequential LLM ingredients arrived through a shift rather than a from-scratch invention: attention was rebuilt from a side-feature into the core architecture, and RLHF was borrowed from robotics and gaming into NLP. The practical implication is that the system should keep the door open for important advances coming from cross-domain transfer or internal reconfiguration, while still recognizing that two examples alone do not prove this is the dominant path forward.