Every AI-generated image you have seen (the surreal art, the fake product shots, the too-perfect stock photos) came out of one of three families of models: GANs, VAEs, or diffusion models. They all solve the same puzzle, which is how to conjure new, believable data out of noise. They just go about it in very different ways, and the one that won was not the one most people bet on a few years ago.
Quickly: a GAN sets two networks against each other, one forging images and one trying to catch the forgeries. A VAE compresses data into a compact code and learns to decode new samples from it. A diffusion model learns to start from pure static and remove noise step by step until an image emerges. In 2026, diffusion is the one running most of the tools you have heard of.
The three approaches compared
| Item | GAN | VAE | Diffusion |
|---|---|---|---|
| Core idea | Generator vs discriminator | Encode then decode | Reverse a noising process |
| Training stability | Notoriously tricky | Stable | Stable |
| Sample quality | Sharp | Often blurry | Sharpest and most varied |
| Generation speed | Fast | Fast | Slower (many steps) |
| Seen in | Early deepfakes, StyleGAN | Research, compression | Midjourney, Stable Diffusion |
GANs: forgers versus detectives
The Generative Adversarial Network, introduced in 2014, is a elegant idea. Two networks train together: a generator tries to produce convincing fake images, and a discriminator tries to tell fakes from real ones. As each improves, it pushes the other to get better, until the generator is producing images the discriminator can no longer reliably catch. For years GANs produced the most notable results, and StyleGAN’s eerily realistic faces went viral for good reason. The drawback is that this adversarial dance is hard to balance: training can collapse, oscillate, or quietly stop improving, and getting it right takes patience and tuning.
VAEs: learning a compact code
A Variational Autoencoder takes a different route. It squeezes each input down into a small “latent” representation and then learns to reconstruct the original from that code. Once trained, you can sample new points from that latent space and decode them into fresh data. VAEs are stable and pleasant to train, and the structured latent space they build is useful. You can move smoothly through it and watch outputs change in sensible ways. Their weakness is sharpness: VAE samples often come out a little soft or blurry compared to the alternatives. They remain valuable, though often as a component inside a larger system rather than the star.
Diffusion: patience beats cleverness
Diffusion models flipped the script by doing something that sounds almost backwards. During training, the model watches images get gradually destroyed by added noise, and it learns to reverse that process, to predict and subtract the noise one small step at a time. To generate, you hand it pure random static and let it denoise its way to a coherent image over many steps.
That step-by-step patience is exactly why diffusion took over. It trains far more stably than a GAN and produces sharper, more diverse results than a VAE. The cost is speed: generating an image means running many denoising steps, which is slower than a GAN’s single pass, though clever sampling tricks have narrowed that gap considerably. The tools most people now associate with AI art, from Midjourney to Stable Diffusion, are diffusion models under the hood.
What the research shows
The original GAN paper by Goodfellow and colleagues (2014) framed generation as a two-player game in which the generator is trained to maximize the probability of the discriminator making a mistake. The authors showed that in theory a unique solution exists, with the generator recovering the training data distribution, and that the whole system can be trained with backpropagation. The VAE came from Kingma and Welling in 2013, whose reparameterization of the variational lower bound made it practical to train models with continuous latent variables on large datasets.
Diffusion models moved to the front in steps. Ho and colleagues (2020) presented high-quality image synthesis with denoising diffusion probabilistic models. In 2021, Dhariwal and Nichol reported that diffusion models can reach image sample quality superior to the generative models that were then state of the art, with FID scores (lower is better) of 2.97 on ImageNet at 128 by 128 pixels, 4.59 at 256 by 256 and 7.72 at 512 by 512. They also matched BigGAN-deep with as few as 25 forward passes per sample, while covering the data distribution better. Their paper is titled “Diffusion Models Beat GANs on Image Synthesis”.
The speed cost described above was then tackled directly. Rombach and colleagues (2021) applied diffusion in the latent space of pretrained autoencoders, so that diffusion models could be trained on limited computational resources while retaining quality, and added cross-attention layers so the model can be conditioned on inputs such as text. Stable Diffusion is built on this latent diffusion approach. For a step-by-step explanation of the process, see our guide to how diffusion models work.
So which one wins?
For high-quality image generation today, diffusion is the clear default. It hits the best balance of quality, variety, and training stability. GANs still have a place where generation speed matters more than variety, and VAEs remain useful for representation learning and as building blocks. But if you are starting a new image project in 2026, you are almost certainly starting with diffusion.
All three lean heavily on convolutional building blocks under the hood, so if you want the foundation, our guide to convolutional neural networks pairs well with this, and the architecture cheat sheet shows how everything connects.
Key takeaways
- GANs introduced the adversarial idea in 2014, and VAEs introduced practical latent-variable training in 2013.
- Diffusion models overtook GANs on benchmark image quality in 2021 and became the base of most image generators.
- Latent diffusion cut the cost of diffusion by working in a compressed space.
Sources
- Goodfellow et al., “Generative Adversarial Networks”, 2014.
- Kingma and Welling, “Auto-Encoding Variational Bayes”, 2013.
- Ho, Jain and Abbeel, “Denoising Diffusion Probabilistic Models”, 2020.
- Dhariwal and Nichol, “Diffusion Models Beat GANs on Image Synthesis”, 2021.
- Rombach et al., “High-Resolution Image Synthesis with Latent Diffusion Models”, 2021.
How we checked this: we compared the figures and findings above with each paper’s published abstract and record. Results are specific to the tasks and datasets those papers used.
