Recurrent Neural Network. An older architecture for sequence processing that maintains hidden state across timesteps. Each step's output feeds back as input to the next. Struggled with long sequences due to vanishing gradients. LSTMs and GRUs improved on this, but transformers ultimately won out.
A neural network architecture where connections form loops, letting the network maintain a form of memory across sequences.
Long Short-Term Memory.
The neural network architecture behind virtually all modern AI language models.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.