beam search
/ˈbiːm ˌsɜːrtʃ/
a search algorithm that explores multiple candidate sequences simultaneously
beam search in a sentence
“Beam search with width 5 tracks the five most promising response paths.”
Origin of beam search
Old English bēam tree, ray of light + Old French cerchier to search
Related Words
greedy decoding
always selecting the highest probability token at each step
top-k sampling
sampling only from the k most likely next tokens
nucleus sampling
sampling from tokens comprising the top cumulative probability mass (top-p)
logits
raw, unnormalized scores output by the model before conversion to probabilities
softmax
a function that converts logits into a probability distribution summing to one
KV cache
cached key-value pairs from previous tokens to speed up autoregressive generation