park-2023-testbed-llama2-7b-decoder-only

IN premise — summaries/2026/08/24/park-2023-linear-representation-s3-inner-product-for-language-model.md

Created 2026-08-24T17:11:03+00:00

Park et al. (2023) use LLaMA-2 7B (a decoder-only Transformer with 32K context window) as their testbed model for empirically validating the linear representation hypothesis and causal inner product.

Summary

Park et al. ground their empirical tests in a single, well-known model (LLaMA-2 7B, a standard decoder-only architecture with a 32K token context), which means their findings about linear representations and causal inner products are specific to that class of model and should be read as evidence for decoder-only Transformers in particular rather than as a universal claim across all architectures. This pins the rest of their argument to a concrete, reproducible experimental setup that others can verify or challenge.