Denoising Diffusion Probabilistic Models
ReadTrains a network to reverse a gradual noising process, with a simple objective that turns out to be a reweighted variational bound. The foundation of modern image generation.
My notes
The VAE paper paid off here — the derivation is an ELBO over a long chain of latents. The final loss, though, is embarrassingly simple: predict the noise.
Forward and Reverse
L_simple = E_{t, x₀, ε} ‖ ε − ε_θ( √ᾱ_t · x₀ + √(1−ᾱ_t) · ε , t ) ‖²Notes
- You can jump straight to any x_t in closed form, so training samples a random t each step — no need to simulate the chain.
- Predicting ε instead of x₀ or the mean made training stable. The paper is honest that this was an empirical choice.
- Sampling needs ~1000 steps. Almost every follow-up (DDIM, latent diffusion, distillation) is about cutting that number.