sampling
/ˈsæmpɫɪŋ/
randomly selecting the next token from the probability distribution rather than always choosing the most likely
sampling in a sentence
“Top-p sampling only considers tokens whose cumulative probability exceeds a threshold.”
Origin of sampling
Old French essample example from Latin exemplum
Related Words
beam search
a search algorithm that explores multiple candidate sequences simultaneously
greedy decoding
always selecting the highest probability token at each step
top-k sampling
sampling only from the k most likely next tokens
nucleus sampling
sampling from tokens comprising the top cumulative probability mass (top-p)
logits
raw, unnormalized scores output by the model before conversion to probabilities
softmax
a function that converts logits into a probability distribution summing to one