Low-Rank Adaptation. An efficient fine-tuning method that freezes the original model weights and only trains small adapter matrices. Drastically reduces the compute and memory needed for fine-tuning — you can customize a 70B model on a single GPU. QLoRA adds quantization for even more savings.
The process of taking a pre-trained model and continuing to train it on a smaller, specific dataset to adapt it for a particular task or domain.
Reducing the precision of a model's numerical values — for example, from 32-bit to 4-bit numbers.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
The research field focused on making sure AI systems do what humans actually want them to do.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.