bias-variance-decomposition-three-terms

IN premise — entries/2026/06/21/wiki-BiasE28093variance_tradeoff-chunk-1.md

Created 2026-06-21T09:55:49+00:00

Expected generalization error under MSE decomposes into exactly three additive terms: squared bias, variance, and irreducible error (noise).

Dependents

These beliefs depend on this one:

• IN ensemble-methods-decompose-bias-variance-independently — Ensemble methods provide complementary and independent controls over the two components of prediction error: bagging (random forests) reduces variance by averaging decorrelated models, while boosting reduces bias by iteratively correcting residuals — together enabling targeted error reduction.
• IN ml-error-decomposition-universal-principle — Error decomposition into irreducible and reducible components appears as a recurring structural pattern across ML paradigms — supervised learning decomposes prediction error into bias + variance + irreducible noise, while reinforcement learning independently decomposes suboptimality into asymptotic bias + overfitting. This parallel suggests that separating what-can-be-learned from what-cannot may be a common structural feature of learning problems, though evidence from only two paradigms is insufficient to establish it as a universal invariant of learning itself.
• IN no-universal-optimal-model — No universally optimal ML model exists: the No Free Lunch theorem establishes this impossibility in principle, and the bias-variance decomposition reveals the mechanism — any fixed model trades bias against variance depending on the problem.