What is a prompt set?
A group of prompts for evaluating a large language model.
prompt set explained in plain English
A group of prompts for evaluating a large language model. For example, the following illustration shows a prompt set consisting of three prompts: Good prompt sets consist of a sufficiently "wide" collection of prompts to thoroughly evaluate the safety and helpfulness of a large language model. See also response set.
Example
Practitioners refer to prompt set when building, training, or evaluating machine learning systems. It appears in research papers, product documentation, and technical discussions about AI capabilities and limitations.
People also read
- average precision at k
A metric for summarizing a model's performance on a single prompt that generates ranked results, such as a numbered list of book recommendations.
- black box model
A model whose "reasoning" is impossible or difficult for humans to understand.
- Chain-of-Thought Prompting
Asking an AI to show its reasoning step by step before giving a final answer, which often improves accuracy on complex tasks.
- Confabulation
When an AI produces a confident, fluent answer that sounds true but is factually wrong — generating plausible language without a reliable link to reality.
- conversational coding
An iterative dialog between you and a generative AI model for the purpose of creating software.
- deterministic
A system that always returns the same output for a given input.
- direct prompting
Synonym for zero-shot prompting.
- distillation
The process of reducing the size of one model (known as the teacher) into a smaller model (known as the student) that emulates the original model's predictions as faithfully as possible.
- emergent behavior
An LLM's ability to generate responses for prompts that it was not explicitly trained on.
- evaluation
The process of measuring a model's quality or comparing different models against each other.