sampling

sampling

/ˈsæmpɫɪŋ/

LLM Inference

randomly selecting the next token from the probability distribution rather than always choosing the most likely

Top-p sampling only considers tokens whose cumulative probability exceeds a threshold.

Origin: Old French essample example from Latin exemplum