perplexity
/pərˈpleksəti/
a measure of how surprised a model is by text; lower means better prediction
perplexity in a sentence
“Perplexity fell steadily as pre-training progressed.”
Origin of perplexity
Latin perplexus entangled, confused
Related Words
benchmark saturation
when models max out a test so it can no longer tell them apart
observability
the degree to which a system's inner behavior can be inferred from what it emits
verbosity bias
a judge's tendency to score longer answers higher regardless of quality
evals
systematic tests to measure model performance on specific tasks
LLM-as-a-Judge
using a strong LLM to evaluate the outputs of another model
ground truth
the actual absolute truth or correct answer used for comparison