What is a denoising?
A common approach to self-supervised learning in which: 1.
denoising explained in plain English
A common approach to self-supervised learning in which: 1. Noise is artificially added to the dataset. 2. The model tries to remove the noise. Denoising enables learning from unlabeled examples. The original dataset serves as the target or label and the noisy data as the input. Some masked language models use denoising as follows: 1. Noise is artificially added to an unlabeled sentence by masking some of the tokens. 2. The model tries to predict the original tokens.
Example
Practitioners refer to denoising when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.
People also read
- average precision at k
A metric for summarizing a model's performance on a single prompt that generates ranked results, such as a numbered list of book recommendations.
- BERT
A model architecture for text representation.
- Character N-gram F-score
A metric to evaluate machine translation models.
- citation precision
A metric that answers the following question: What percentage of the citations in an LLM's response were actually correct and supportive?
- citation recall
A metric that answers the following question: What percentage of the source documents the LLM used to compose its response are actually cited in the response?
- cross-entropy
A generalization of Log Loss to multi-class classification problems.
- depth
The sum of the following in a neural network: - the number of hidden layers - the number of output layers, which is typically 1 - the number of any embedding layers For example, a neural network with five hidden layers and one output layer has a depth of 6.
- Embedding
A numerical representation of text, images, or other data that captures semantic meaning.
- embedding layer
A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimension embedding vector.
- encoder
In general, any ML system that converts from a raw, sparse, or external representation into a more processed, denser, or more internal representation.