park-2025-set-inclusion-spurious-orthogonality

IN premise — summaries/2026/08/24/park-2024-categorical-hierarchical-concepts-sR-references-chunk-1.md

Created 2026-08-25T02:58:25+00:00

Shuffled unembeddings reproduce orthogonality between child-parent and parent vectors purely from set inclusion (Y(z) ⊆ Y(w)), not semantic meaning, demonstrating that set inclusion alone is insufficient to explain the geometric structure.

Summary

When the parts of these vector embeddings are randomly shuffled, the same perpendicular angles between related and parent vectors still show up, but only because one set of items happens to be contained inside another, not because of any actual meaning. In other words, the geometric structure that looks like it encodes semantic relationships is just an artifact of simple subset membership, so angles and directions in these vectors cannot be trusted as evidence of real conceptual connections.

Dependents

These beliefs depend on this one:

OUT cross-model-convergence-conditional-on-artifact-control — The cross-model convergence of geometric structure (polytope geometry, feature universality, orthogonality) constitutes a genuine architectural property, but the orthogonality component specifically requires set-inclusion controls to distinguish genuine hierarchical semantics from combinatorial artifacts.
OUT geometric-framework-not-artifact — The full geometric framework of LLMs (covariance whitening, polytope decomposition, Riesz isomorphism, cross-model convergence) reflects genuine architectural structure rather than a mathematical artifact of the analysis method, provided the underlying orthogonality is not reducible to trivial set-inclusion between parent and child token sets.
OUT hierarchical-orthogonality-genuine-structure — The observed parent⊥(child−parent) orthogonality in LLM representation spaces reflects genuine hierarchical semantics rather than a trivial set-inclusion artifact.
IN shuffling-contradiction-resolution — The apparent contradiction between the 2025 empirical control (shuffling unembeddings *destroys* the full orthogonality structure) and the set-inclusion finding (shuffled unembeddings *reproduce* the child-parent⊥parent sub-orthogonality) is resolved by scope: the *full* four-condition Theorem 8 structure is genuine and shuffle-destroyed, while a *specific* sub-condition (consecutive-level parent⊥child-parent) is a set-inclusion artifact—meaning the geometric framework captures the genuine full structure, not the spurious sub-structure.