error analysis
/ˈerər əˌnæləsɪs/
systematically reading failures to find the patterns worth fixing
error analysis in a sentence
“Error analysis showed half the misses came from date formats.”
Origin of error analysis
Latin errare to wander + Greek analusis a loosening
Related Words
perplexity
a measure of how surprised a model is by text; lower means better prediction
benchmark saturation
when models max out a test so it can no longer tell them apart
observability
the degree to which a system's inner behavior can be inferred from what it emits
verbosity bias
a judge's tendency to score longer answers higher regardless of quality
evals
systematic tests to measure model performance on specific tasks
LLM-as-a-Judge
using a strong LLM to evaluate the outputs of another model