sae-golden-gate-bridge-neighborhood-structure
IN premise — summaries/2026/08/24/templeton-2024-scaling-monosemanticity-chunk-5.md
Created 2026-08-25T02:58:38+00:00
Features near the Golden Gate Bridge feature (34M/31164353) include San Francisco locations (Alcatraz, Presidio), California landmarks (Lake Tahoe, Yosemite, Solano County), and further-out tourist attractions (Médoc wine region, Isle of Skye).
Summary
The model organizes its geographic and tourist concepts in a coherent neighborhood: sitting near the Golden Gate Bridge in the feature space, you find other San Francisco spots, broader California landmarks, and even distant tourist destinations, forming a loose hierarchy from local to international. This matters because it confirms the representation is semantically structured in a way that mirrors how humans group places, which makes the model's internal geometry interpretable and predictable rather than arbitrary.
Dependents
These beliefs depend on this one:
- OUT feature-hierarchy-empirical-validation — The hierarchical feature structure predicted geometrically by Park's orthogonality theorem (parent⊥child−parent) is empirically instantiated in SAE decoder-space neighborhoods: abstract/general features (transit infrastructure) subsume concrete/specific features (Golden Gate Bridge, Alcatraz), and this neighborhood hierarchy mirrors the WordNet synset hierarchy that Park validates across Gemma-2B and LLaMA-3-8B.