masked language model

A language model that predicts the probability of candidate tokens to fill in blanks in a sequence. For example, a masked language model can calculate probabilities for candidate word(s) to replace the underline in the following sentence: The ____ in the hat came back. The literature typically uses the string "MASK" instead of an underline. For example: The "MASK" in the hat came back. Most modern masked language models are bidirectional.

Practitioners refer to masked language model when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.