rome-applicable-autoregressive-decoder

IN premise — summaries/2026/08/24/meng-2022-rome-s6-ethical-considerations.md

Created 2026-08-25T02:58:16+00:00

ROME is applicable to autoregressive (decoder-only) transformers, in contrast to several prior editing methods that focus on masked LMs (BERT) or seq2seq models (BART, T5).

Summary

This confirms that ROME works on the GPT-style architectures that power most modern chatbots and language tools, not just the older encoder or translation-style models that earlier editing techniques targeted. Practically, it means you can surgically correct a specific fact in a mainstream LLM without retraining, which is a capability that simply wasn't available for the dominant model class before this result.