golden dataset
/ˈɡoʊldən ˈdeɪtəset/
a hand-verified set of examples used as the standard for judging model output
golden dataset in a sentence
“Every release is scored against the golden dataset before it ships.”
Origin of golden dataset
golden in the sense of gold standard, from gold-backed currency
Related Words
rubric
an explicit scoring guide that turns judgment into repeatable criteria
pass@k
the probability that at least one of k sampled attempts is correct
drift
gradual change in inputs or behavior that quietly erodes performance
error analysis
systematically reading failures to find the patterns worth fixing
perplexity
a measure of how surprised a model is by text; lower means better prediction
benchmark saturation
when models max out a test so it can no longer tell them apart