Few-Shot Learning

Definition

The ability of a model to learn a new task from just a handful of examples, often provided in the prompt itself. Large language models excel at this — show them 2-3 examples of a task and they can generalize. Contrasts with traditional ML that needs thousands of labeled examples.

How It Works

Few-shot learning is when a model learns to perform a task from just a handful of examples, rather than needing thousands or millions. In practice, this usually means including 2-5 examples in your prompt to show the model what you want. "Here's how I want you to format this data: [example 1], [example 2]. Now do the same for this input."

This capability is one of the most practically useful features of large language models. Before LLMs, building a classifier or text converter meant collecting a labeled dataset, training a model, tuning hyperparameters, and deploying it. Now you can get 80% of the way there by showing a few examples in a prompt. It's not as robust as a dedicated model, but the speed of iteration is transformative.

Few-shot learning sits on a spectrum with zero-shot (no examples, just instructions) and many-shot (lots of examples). The sweet spot depends on the task complexity. For simple formatting tasks, zero-shot often works fine. For nuanced judgment calls — like classifying customer complaints into specific categories your business uses — a few well-chosen examples make a huge difference. The key is picking representative examples that cover the edge cases.