logits
/ˈloʊdʒɪts/
raw, unnormalized scores output by the model before conversion to probabilities
logits in a sentence
“Logits are converted to probabilities using the softmax function.”
Origin of logits
From log-odds in statistics, coined 1944
Related Words
softmax
a function that converts logits into a probability distribution summing to one
KV cache
cached key-value pairs from previous tokens to speed up autoregressive generation
inference
the process of using a trained model to generate predictions or outputs
temperature
a parameter controlling randomness in generation—higher means more creative, lower means more deterministic
sampling
randomly selecting the next token from the probability distribution rather than always choosing the most likely
beam search
a search algorithm that explores multiple candidate sequences simultaneously