logits
/ˈloʊdʒɪts/
raw, unnormalized scores output by the model before conversion to probabilities
logits in a sentence
“Logits are converted to probabilities using the softmax function.”
Origin of logits
From log-odds in statistics, coined 1944
Related Words
softmax
a function that converts logits into a probability distribution summing to one
KV cache
cached key-value pairs from previous tokens to speed up autoregressive generation
inference
the process of using a trained model to generate predictions or outputs
temperature
a sampling parameter that flattens or sharpens the next-token probability distribution
sampling
randomly selecting the next token from the probability distribution rather than always choosing the most likely
beam search
a search algorithm that explores multiple candidate sequences simultaneously