Training a smaller model to replicate the behavior of a larger one.
Training a smaller model to replicate the behavior of a larger one. The small model learns from the bigger model's probability distributions rather than raw data, capturing nuanced knowledge more efficiently. How companies create fast, cheap models that still perform well.
A technique where a smaller 'student' model learns to mimic a larger 'teacher' model.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
An autonomous AI system that can perceive its environment, make decisions, and take actions to achieve goals.
The research field focused on making sure AI systems do what humans actually want them to do.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.