rater

A human who provides labels for examples. "Annotator" is another name for rater. See Categorical data: Common issues in Machine Learning Crash Course for more information.

Practitioners refer to rater when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.