rlhf-origin-atari-not-nlp

IN premise — entries/2026/06/21/wiki-Reinforcement_learning_from_human_feedback.md

Created 2026-06-21T09:50:10+00:00

RLHF was originally applied to Atari game bots and simulated robotics (Christiano et al., 2017) before being adapted for natural language processing.

Summary

RLHF was first designed to teach game-playing bots and simulated robots to follow human preferences, not to shape language models. This matters because the technique's reward structures, exploration logic, and evaluation methods were all originally tuned for environments with clear win/loss states, so its use in open-ended text generation is an adaptation that may carry over assumptions that don't fit as cleanly.

Dependents

These beliefs depend on this one:

IN rlhf-transferred-from-games-via-universal-evaluation-property — RLHF transferred from Atari/robotics (Christiano 2017) to NLP because its motivating property — tasks that are hard to specify but easy to judge — is domain-independent, enabling the same preference-comparison framework to align game bots and language models alike.