What does the term "latent variable" refer to in generative models?

The term "latent variable" in generative models refers to an underlying factor that the model learns to represent. Latent variables are not directly observed but are inferred from the data. They capture hidden structures or properties that influence the observed variables. For instance, in a generative model for images, latent variables might represent concepts like style, shape, or color, which are not explicitly labeled in the dataset but play a crucial role in generating new, coherent samples that exhibit similar characteristics.

This underlying representation allows the model to generalize and generate diverse outputs by manipulating these latent variables. It emphasizes how models can extract meaningful patterns from complex data distributions, facilitating tasks like data imputation, clustering, and generating novel content that aligns with learned attributes. As such, latent variables are fundamental in understanding the internal workings of generative models and how they relate to observable data.