judge bias
/ˈdʒʌdʒ baɪəs/
a systematic preference in a human or model evaluator that distorts scores independently of quality
judge bias in a sentence
“Randomizing answer order reduced the judge bias toward the first response.”
Origin of judge bias
Latin judicare to form an opinion + Old French biais slant
Related Words
hallucination
fluent model output that is false, unsupported, or inconsistent with the supplied evidence
sycophancy
a model's tendency to agree with a user's stated view instead of giving an accurate independent answer
unsupported claim
a statement for which the available evidence does not provide adequate support
verification
checking an output against independent evidence, calculations, tests, or authoritative sources
eval
a structured test used to measure how well a model or AI system performs a defined task
rubric
an explicit set of criteria and scoring rules for judging an output