What is a language model?
A model that estimates the probability of a token or sequence of tokens occurring in a longer sequence of tokens.
language model explained in plain English
A model that estimates the probability of a token or sequence of tokens occurring in a longer sequence of tokens.
Though counterintuitive, many models that evaluate text are not language models. For example, text classification models and sentiment analysis models are not language models. --- See What is a language model? in Machine Learning Crash Course for more information.
Example
Practitioners refer to language model when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.
People also read
- encoder
In general, any ML system that converts from a raw, sparse, or external representation into a more processed, denser, or more internal representation.
- bag of words
A representation of the words in a phrase or passage, irrespective of order.
- BERT
A model architecture for text representation.
- Character N-gram F-score
A metric to evaluate machine translation models.
- cross-entropy
A generalization of Log Loss to multi-class classification problems.
- Embedding
A numerical representation of text, images, or other data that captures semantic meaning.
- embedding layer
A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimension embedding vector.
- fraction of successes
A metric for evaluating an ML model's generated text.
- Inference
The phase when a trained model is actually used — taking new input and producing a prediction or response.
- perplexity
One measure of how well a model is accomplishing its task.