gan-implicit-nature-explains-training-difficulty

IN derived (depth 2)

Created 2026-06-21T11:39:46+00:00 · Reviewed 2026-06-21T15:37:01+00:00

GANs' implicit generative nature (no explicit likelihood function) is plausibly connected to their need for multiple complementary training interventions — without a tractable objective to optimize directly, training stability relies on several distinct design choices (non-saturating loss, TTUR, deterministic discriminators) each addressing a different failure mode, which may partly substitute for the more direct optimization signal that explicit-likelihood models enjoy.

Justifications

SL — Implicit likelihood-free formulation necessitates the multiple stabilization interventions

Antecedents (all must be IN):

IN gan-implicit-generative-model — GANs are implicit generative models that do not explicitly model the likelihood function and cannot find the latent variable for a given sample, unlike flow-based models or VAEs
IN gan-training-stability-requires-multiple-interventions — GAN training stability benefits from several complementary design choices — non-saturating loss to address vanishing gradients when the generator is poor, two-timescale update rule for provable convergence to a stationary local Nash equilibrium (though not mode collapse prevention), and deterministic discriminators for optimality of the discriminator — each addressing a distinct failure mode, but none individually sufficient for overall stability.

Dependents

These beliefs depend on this one:

IN gan-exemplifies-pragmatism-crisis-at-model-level — GANs recapitulate at the individual model level the field-wide pattern where pragmatic shortcuts drive both capability and crisis — their implicit generative approach (pragmatically avoiding intractable likelihood computation) simultaneously enabled unique capabilities (single-pass generation, cross-domain applications) and created fundamental training instability, making GANs the clearest single-model exemplar of the pragmatism-crisis dynamic.