What is a cross-entropy?
A generalization of Log Loss to multi-class classification problems.
cross-entropy explained in plain English
A generalization of Log Loss to multi-class classification problems. Cross-entropy quantifies the difference between two probability distributions. See also perplexity.
Example
Practitioners refer to cross-entropy when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.
People also read
- embedding layer
A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimension embedding vector.
- encoder
In general, any ML system that converts from a raw, sparse, or external representation into a more processed, denser, or more internal representation.
- fraction of successes
A metric for evaluating an ML model's generated text.
- Inference
The phase when a trained model is actually used — taking new input and producing a prediction or response.
- language model
A model that estimates the probability of a token or sequence of tokens occurring in a longer sequence of tokens.
- perplexity
One measure of how well a model is accomplishing its task.
- AUC
A number between 0.
- average precision at k
A metric for summarizing a model's performance on a single prompt that generates ranked results, such as a numbered list of book recommendations.
- Backpropagation
The process that tells a neural network which internal settings caused an error and how to adjust them, working backwards through layers.
- bag of words
A representation of the words in a phrase or passage, irrespective of order.