cnn-efficiency-from-inductive-biases
IN derived (depth 1)
Created 2026-06-21T09:59:01+00:00 · Reviewed 2026-06-21T15:37:01+00:00
CNNs achieve dramatic parameter efficiency through two complementary inductive biases — local connectivity restricts each neuron's receptive field, and weight sharing forces all spatial positions to use identical filters — reducing parameters by orders of magnitude.
Justifications
SL — Local connectivity and weight sharing jointly produce orders-of-magnitude parameter reduction
Antecedents (all must be IN):
- IN cnn-local-connectivity — Convolutional layers differ from fully connected layers by connecting each neuron to only a local subset of neurons in the previous layer rather than all neurons
- IN cnn-parameter-sharing-translation-invariance — Parameter sharing in convolutional layers (all neurons in a depth slice share weights and bias) reduces free parameters and provides translation invariance
- IN cnn-weight-sharing-parameter-reduction — CNN weight sharing reduces parameters dramatically: a 5x5 kernel uses 25 weights vs 10,000 for a fully connected layer on 100x100 input
Dependents
These beliefs depend on this one:
- IN cnn-pragmatic-not-faithful-biological-inspiration — CNNs demonstrate that effective ML architectures don't require biological fidelity — their inductive biases (local connectivity, weight sharing) capture the right structural constraints for dramatic parameter efficiency, even though the "convolution" is technically cross-correlation and the equivariance doesn't produce true invariance.
- IN manifold-structure-explains-inductive-bias-effectiveness — The manifold hypothesis offers a theoretical lens for understanding why certain architectural inductive biases are effective — if high-dimensional data lies on low-dimensional manifolds (and prediction is formally equivalent to compression of that structure), then architectures exploiting local connectivity and weight sharing can be seen as responses to data geometry rather than purely ad hoc engineering choices.
- IN spatial-vs-relational-inductive-bias-paradigms — CNNs and Transformers represent two different inductive bias strategies — CNNs achieve parameter efficiency through local connectivity and weight sharing, while Transformers build on multi-head self-attention as a unified mechanism adaptable to distinct task families — suggesting different architectural strengths, though the specific claim that CNNs dominate fixed-topology data and Transformers dominate variable-length sequences goes beyond what these antecedents establish.