spatial-vs-relational-inductive-bias-paradigms

IN derived (depth 2)

Created 2026-06-21T10:13:05+00:00 · Reviewed 2026-06-21T15:37:01+00:00

CNNs and Transformers represent two different inductive bias strategies — CNNs achieve parameter efficiency through local connectivity and weight sharing, while Transformers build on multi-head self-attention as a unified mechanism adaptable to distinct task families — suggesting different architectural strengths, though the specific claim that CNNs dominate fixed-topology data and Transformers dominate variable-length sequences goes beyond what these antecedents establish.

Justifications

SL — depth-2 — the two dominant architectures encode complementary geometric assumptions about data, explaining their different domains of dominance

Antecedents (all must be IN):

IN cnn-efficiency-from-inductive-biases — CNNs achieve dramatic parameter efficiency through two complementary inductive biases — local connectivity restricts each neuron's receptive field, and weight sharing forces all spatial positions to use identical filters — reducing parameters by orders of magnitude.
IN transformer-architecture-unified-yet-specialized — The Transformer is simultaneously a unified architecture (all variants built on multi-head self-attention) and a specialized one (encoder-only, decoder-only, encoder-decoder serve distinct task families).

Dependents

These beliefs depend on this one:

OUT manifold-geometry-explains-inductive-bias-spectrum — The manifold hypothesis unifies the CNN-Transformer inductive bias spectrum — CNNs exploit local manifold geometry through spatial locality and weight sharing, while Transformers exploit global manifold structure through permutation-invariant attention, representing complementary strategies for navigating the same underlying low-dimensional data geometry.