With the rise of large language models such as GPT, generative models have become a cornerstone of modern Artificial Intelligence. However, GPT and its counterparts represent just one branch of a broader generative modeling family. This article breaks down the landscape into two primary categories: explicit density models, which calculate or approximate the probability of each generated sample, and implicit density models, which focus on producing realistic data without direct probability estimates.
Explicit density models are divided into two classes—tractable and approximate. Tractable models like autoregressive networks (e.g., GPT, PixelCNN) and normalizing flow models allow exact, efficient probability calculation for any generated example. These models excel in applications where understanding the likelihood of data points is critical. Approximate models, including variational autoencoders (VAEs), energy-based models, and diffusion models, cannot compute exact probabilities and instead rely on clever mathematical approximations, such as the evidence lower bound in VAEs. This distinction is vital when use cases require either strict statistical guarantees or are tolerant of statistical estimation.
Implicit density models, in contrast, generate realistic data without computing sample probabilities. Generative adversarial networks (GANs) exemplify this approach, where a generator aims to fool a discriminator by producing synthetic samples; this rivalry enables highly convincing outputs, especially in image synthesis. Notable GAN variants include Conditional GANs, CycleGAN for style transfer, StyleGAN for fine-grained control over generated content, and BigGAN for scaling to high-quality, diverse imagery. Score-based generative models offer another implicit approach, iteratively transforming noise into realistic data by following learned gradients that guide samples toward high-density regions, without knowing their explicit probabilities.
The article concludes by contextualizing the strengths and trade-offs between explicit and implicit models. Explicit models like transformers underpin much of today’s successful practical Artificial Intelligence deployments due to their statistical rigor and flexibility. Implicit models continue to innovate in generating lifelike images and other content, though typically with less control over statistical evaluation. The overall generative model map illustrates the richness and ongoing evolution in how machines learn to create, offering both structured probability and creative realism.