architecture-taxonomy-recapitulates-data-structure-hierarchy

IN derived (depth 5)

Created 2026-06-21T11:27:22+00:00 · Reviewed 2026-06-21T15:37:01+00:00

The neural network architecture taxonomy (MLP → CNN → RNN/LSTM → Transformer → Mamba) mirrors a hierarchy of data structure assumptions — MLPs assume no structure, CNNs assume local spatial structure, RNNs assume sequential structure, Transformers assume global relational structure, and each architecture's effectiveness is explained by matching its inductive bias to the manifold geometry of its target data type.

Justifications

SL — The architecture taxonomy is not arbitrary progression but reflects increasingly sophisticated assumptions about data manifold structure, with effectiveness governed by geometry matching

Antecedents (all must be IN):

IN neural-arch-taxonomy-mlp-cnn-rnn-transformer-mamba — Neural network architecture taxonomy progresses: MLP (feedforward) → CNN (spatial) → RNN/LSTM/GRU (sequential) → Transformer (attention-based, now dominant) → Mamba (state-space, emerging alternative).
IN cnn-not-limited-to-images — CNNs are not limited to image data — they apply wherever local spatial or temporal structure exists, including audio (time domain), spectrograms (frequency domain), time series, and text
IN effective-architectures-are-geometry-matched-compression — Since prediction and compression are formally equivalent, and data geometry (the manifold hypothesis) offers a geometric explanation for why certain architectural inductive biases succeed, effective ML architectures can be understood as implicit compression algorithms whose success depends on alignment with the data's intrinsic geometry. CNNs exploit spatial locality and transformers exploit relational structure, and this framework suggests they succeed when the data's geometric properties match their compression strategy — though the formal connection between manifold geometry and the prediction-compression equivalence remains conceptual rather than proven.

Dependents

These beliefs depend on this one:

IN architecture-taxonomy-independently-validates-manifold — The neural architecture taxonomy's organization by data structure (MLP for unstructured → CNN for spatial → RNN for sequential → Transformer for relational) provides supporting evidence for the manifold hypothesis as a theoretical anchor — the fact that architectures can be organized by the geometry they exploit is consistent with the claim that data geometry is a fundamental organizing principle, and this coherence between the taxonomy pattern and the theoretical framework strengthens both.