activation
/ˌæktəˈveɪʃən/
the output of a neuron after applying a non-linear function
activation in a sentence
“ReLU activation introduces non-linearity to the network.”
Origin of activation
Latin activus (active) + -ation
Related Words
gradient
the direction and rate of steepest increase of a function
inference
using a trained model to make predictions on new data
fine-tuning
adapting a pre-trained model for a specific task
context window
the maximum amount of text a model can process at once
softmax
a function converting raw scores into a probability distribution
transformer
a neural network architecture using self-attention for sequence processing