A neural network architecture with two parts: an encoder that processes the input into a representation, and a decoder that generates the output from that representation.
A neural network architecture with two parts: an encoder that processes the input into a representation, and a decoder that generates the output from that representation. T5 and BART are encoder-decoder transformers. Good for tasks where the output differs significantly from the input, like translation and summarization.
The part of a neural network that processes input data into an internal representation.
The part of a neural network that generates output from an internal representation.
The neural network architecture behind virtually all modern AI language models.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.