distillation
/ˌdɪstəˈɫeɪʃən/
Training a smaller model to mimic a larger one
distillation in a sentence
“Model distillation created a faster version suitable for mobile devices.”
Origin of distillation
From Latin distillare (to drip down), from de- (down) + stillare (to drip)
Related Words
quantization
Reducing model precision to decrease size and increase speed
latent space
A representation of compressed data
modality
A particular mode in which something exists or is experienced or expressed
large language model
An AI trained on vast text data to understand and generate language
foundation model
A large model trained on broad data that can be adapted to many tasks
context window
The amount of text a model can consider at once