Bidirectional Encoder Representations from Transformers. A landmark model from Google (2018) that reads text in both directions simultaneously, giving it better understanding of context. Revolutionized NLP tasks like search, question answering, and text classification. The foundation for many later models.
The neural network architecture behind virtually all modern AI language models.
A pre-training technique where random words in text are hidden (masked) and the model learns to predict them from context.
Natural Language Processing.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.