self-supervised learning
/ˌself ˌsuːpərˌvaɪzd ˈlɜːrnɪŋ/
training where labels are derived from the data itself, like predicting masked words
self-supervised learning in a sentence
“Self-supervised learning on next-token prediction requires no human labeling.”
Origin of self-supervised learning
Greek auto- self + Latin super- over + videre to see
Related Words
loss function
a mathematical measure of how wrong the model's predictions are, minimized during training
gradient descent
an optimization algorithm that iteratively adjusts parameters to minimize loss
backpropagation
the algorithm for computing gradients by propagating errors backward through the network
overfitting
when a model memorizes training data rather than learning generalizable patterns
regularization
techniques to prevent overfitting by constraining model complexity
pre-training
initial training on vast text data to learn language patterns before task-specific fine-tuning