What is a generative model?
Practically speaking, a model that does either of the following: - Creates (generates) new examples from the training dataset.
generative model explained in plain English
Practically speaking, a model that does either of the following: - Creates (generates) new examples from the training dataset. For example, a generative model could create poetry after training on a dataset of poems. The generator part of a generative adversarial network falls into this category. - Determines the probability that a new example comes from the training set, or was created from the same mechanism that created the training set. For example, after training on a dataset consisting of English sentences, a generative model could determine the probability that new input is a valid English sentence. A generative model can theoretically discern the distribution of examples or particular features in a dataset. That is:
Unsupervised learning models are generative. Contrast with discriminative models.
Example
Practitioners refer to generative model when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.
People also read
- Backpropagation
The process that tells a neural network which internal settings caused an error and how to adjust them, working backwards through layers.
- Bayesian neural network
A probabilistic neural network that accounts for uncertainty in weights and outputs.
- discriminative model
A model that predicts labels from a set of one or more features.
- embedding layer
A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimension embedding vector.
- full softmax
Synonym for softmax.
- gradient boosting
A training algorithm where weak models are trained to iteratively improve the quality (reduce the loss) of a strong model.
- Gradient Descent
The method by which a model gradually improves by making small adjustments after each mistake, moving toward better performance.
- input layer
The layer of a neural network that holds the feature vector.
- logistic regression
A type of regression model that predicts a probability.
- minimax loss
A loss function for generative adversarial networks, based on the cross-entropy between the distribution of generated data and real data.