What is a citation precision?
A metric that answers the following question: What percentage of the citations in an LLM's response were actually correct and supportive?
citation precision explained in plain English
A metric that answers the following question: What percentage of the citations in an LLM's response were actually correct and supportive? That is, what percent of the citations contain the exact facts or relevant information required to verify the claim made in an LLM's response. For example, if an LLM response cited 10 documents, but only 7 of those citations were correct and supportive, then the citation precision would be 0.7.
Example
Practitioners refer to citation precision when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.
People also read
- average precision at k
A metric for summarizing a model's performance on a single prompt that generates ranked results, such as a numbered list of book recommendations.
- BERT
A model architecture for text representation.
- Character N-gram F-score
A metric to evaluate machine translation models.
- citation recall
A metric that answers the following question: What percentage of the source documents the LLM used to compose its response are actually cited in the response?
- cross-entropy
A generalization of Log Loss to multi-class classification problems.
- denoising
A common approach to self-supervised learning in which: 1.
- depth
The sum of the following in a neural network: - the number of hidden layers - the number of output layers, which is typically 1 - the number of any embedding layers For example, a neural network with five hidden layers and one output layer has a depth of 6.
- Embedding
A numerical representation of text, images, or other data that captures semantic meaning.
- embedding layer
A special hidden layer that trains on a high-dimensional categorical feature to gradually learn a lower dimension embedding vector.
- encoder
In general, any ML system that converts from a raw, sparse, or external representation into a more processed, denser, or more internal representation.