A text generation method (also called nucleus sampling) that only considers tokens whose cumulative probability exceeds a threshold P.
A text generation method (also called nucleus sampling) that only considers tokens whose cumulative probability exceeds a threshold P. With top-p of 0.9, the model chooses from the smallest set of tokens that together have a 90% probability. More dynamic than top-k sampling and widely used in practice.
A parameter that controls the randomness of a language model's output.
The process of selecting the next token from the model's predicted probability distribution during text generation.
A mathematical function applied to a neuron's output that introduces non-linearity into the network.
An optimization algorithm that combines the best parts of two other methods — AdaGrad and RMSProp.
Artificial General Intelligence.
An autonomous AI system that can perceive its environment, make decisions, and take actions to achieve goals.
Browse our complete glossary or subscribe to our newsletter for the latest AI news and insights.