The part of a neural network that processes input data into an internal representation.
The part of a neural network that processes input data into an internal representation. In transformers, the encoder reads the full input and creates contextual representations. BERT is an encoder-only model. Encoder-decoder architectures like T5 use both halves for tasks like translation.
The part of a neural network that generates output from an internal representation.
The neural network architecture behind virtually all modern AI language models.
Bidirectional Encoder Representations from Transformers.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.