generative-models-provide-unlimited-training-data

OUT derived (depth 1)

Created 2026-06-21T11:27:22+00:00

Generative models (GANs, VAEs, diffusion) would provide effectively unlimited synthetic training data — generating privacy-preserving medical images, augmenting scarce datasets, and enabling training without data collection barriers — fundamentally resolving the labeled-data bottleneck that constrains supervised learning.

Justifications

SL — Generative models would solve data scarcity unless model collapse from recursive synthetic training or GAN training instability undermine the quality of generated data

Antecedents (all must be IN):

IN generative-model-families-autoregressive-diffusion-gan-vae — Four main generative model families: autoregressive (GPT-style), diffusion (Stable Diffusion), adversarial (GAN), and variational (VAE).
IN gan-synthetic-medical-imaging-privacy — GANs generate synthetic medical images (MRI, PET) to overcome patient privacy barriers that limit access to real medical imaging data
IN transfer-learning-standard-for-small-data — Transfer learning (pretraining on a large dataset then fine-tuning on a small target dataset) is the standard technique when training data is limited, preventing overfitting

Unless (any of these IN defeats this justification):

IN ml-model-collapse-synthetic-data — Model collapse is the degradation that occurs when models train on uncurated synthetic data or outputs of prior model versions, also called 'model autophagy disorder (MAD)'
IN gan-mode-collapse-vs-vanishing-gradient — Mode collapse (generator produces limited modes) and vanishing gradient (generator cannot learn) are opposite GAN failure modes caused by the discriminator being too weak vs. too strong, respectively