Skip to content
Back to search results

Few-shot Learning

Introduced by
Lake, B. M.; Vinyals, O.; Finn, C. et al.
Year
2011–2017
Type
Meta-learning / low-data learning paradigm
Family
Machine learning
Font size

What does it do?

Few-shot learning is a machine learning paradigm that trains models to recognize new classes or solve new tasks from only a handful of labeled examples — typically one to five — by leveraging prior knowledge acquired from a large, related training distribution.

It is especially relevant in domains where labeling is expensive, scarce, or structurally limited.

The intuition behind the method

Humans can reliably identify a new species of bird from a single photograph, because they already carry rich visual knowledge from years of experience. Few-shot learning attempts to replicate this ability computationally: a model is first exposed to many diverse tasks so it learns how to learn quickly, then at inference time it adapts to an entirely new task using only a tiny labeled support set. The key insight is that the right representation and adaptation strategy can substitute for the large labeled dataset that standard supervised learning requires.

History

Few-shot learning gained formal attention around 2011 when Lake et al. demonstrated human-level one-shot learning of handwritten characters using a hierarchical Bayesian program induction model. The field accelerated after 2016 with deep metric-learning approaches — Vinyals et al.'s Matching Networks and Snell et al.'s Prototypical Networks — followed by Finn et al.'s model-agnostic meta-learning (MAML) in 2017, which unified the paradigm under gradient-based fast adaptation. Large pre-trained models have since made few-shot learning a standard capability in modern NLP and vision systems.

When to use it

Use few-shot learning when labeled data for a target task is genuinely scarce — fewer than roughly 10 to 20 examples per class — but a large related dataset is available for pre-training or meta-training. Typical use cases include rare-disease classification, endangered-species recognition, low-resource language tasks, and personalized recommendation with cold-start users. Do not use it as a substitute for collecting more labeled data when labeling is feasible; with sufficient labels, fully supervised or fine-tuned transfer learning will nearly always outperform few-shot methods. Also avoid it if the meta-training distribution is very different from the target domain, as this invalidates the learned prior.

Limitations and considerations

  • Requires a large, diverse meta-training set; performance degrades sharply when the meta-training and target distributions diverge significantly.
  • Optimization-based methods (e.g., MAML) are computationally intensive during meta-training due to second-order gradients.
  • Benchmark accuracy on standard datasets (miniImageNet, Omniglot) can overstate real-world performance if the test domain differs.
  • Choosing the right architecture, embedding backbone, and episode sampling strategy involves non-trivial hyperparameter decisions.
  • With more than about 20 labeled examples per class, standard transfer learning or fine-tuning typically offers better accuracy with less complexity.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Enables practical learning from one to five labeled examples per class, dramatically reducing annotation cost.
  • Episodic training explicitly prepares the model for the low-data deployment scenario it will face.
  • Metric-based variants (Prototypical Networks, Matching Networks) are computationally lightweight at inference — no gradient steps required.
  • Widely applicable across vision, NLP, audio, and structured data tasks.
  • Naturally integrates with large pre-trained models, allowing powerful priors to be exploited without full fine-tuning.

Applications

  • Rare-disease or anomaly classification in medical imaging where annotated cases are inherently scarce.
  • Low-resource natural language processing tasks such as intent detection in new languages or domains.
  • Cold-start personalization in recommendation systems when a new user has very few interactions.
  • Endangered-species or fine-grained species recognition in ecology and biodiversity monitoring.
  • Industrial defect detection when only a handful of defective samples are available for training.
  • Rapid prototyping of new classification tasks in robotics and interactive machine learning.

How it works

How it works

1

Training is organized into episodes, each simulating a few-shot scenario. In every episode a small set of classes is sampled from the training pool, and the model must learn to distinguish them. This episodic training ensures the model's prior knowledge is structured for rapid adaptation rather than rote memorization of specific classes.

2

Each episode divides its examples into a support set (the few labeled examples the model may use) and a query set (held-out examples used to measure how well the model generalized). A k-shot, n-way task provides k labeled examples for each of n novel classes. This structure mirrors the deployment scenario at training time.

3

The model learns an embedding function — typically a deep neural network — that maps inputs to a feature space where same-class examples cluster together and different-class examples are separated. The quality of this shared representation is the primary determinant of downstream few-shot accuracy.

4

At test time the model either computes similarities between query embeddings and support embeddings (metric-based methods such as Matching Networks or Prototypical Networks) or performs a small number of gradient steps to adapt its parameters to the support set (optimization-based methods such as MAML). Both strategies exploit the learned representation without retraining from scratch.

5

The loss on the query set is used to update the meta-learner's parameters across episodes, so that the representation and adaptation strategy improve globally. At deployment, the final model adapts to any new support set in seconds, making predictions on unseen query examples without further optimization.

Common pitfalls

  • Conflating few-shot learning with zero-shot learning: few-shot requires at least one labeled example per novel class at test time.
  • Evaluating on test classes seen during meta-training, inflating reported accuracy.
  • Ignoring domain gap: a model meta-trained on ImageNet will not automatically generalize to medical histology without domain adaptation.
  • Using a standard supervised loss without episodic training and expecting few-shot generalization.
  • Reporting only top-1 accuracy without confidence intervals over multiple episode draws, which masks variance in few-shot results.

Frequently asked questions

What is the difference between few-shot learning and transfer learning?

Transfer learning fine-tunes a pre-trained model on a new task and typically benefits from tens to thousands of labeled examples. Few-shot learning is specifically designed to adapt with only one to five examples per class, using episodic meta-training so the model learns how to generalize quickly rather than how to fit a specific domain.

What does k-shot n-way mean?

It describes the episode structure: n-way means n novel classes are presented simultaneously, and k-shot means k labeled examples are provided per class. A 5-way 1-shot task asks the model to classify query images into one of five categories given only a single labeled example for each.

Do I need a GPU to run few-shot learning?

Meta-training — especially optimization-based methods like MAML — requires substantial compute and is typically done on GPUs. However, metric-based inference (comparing query embeddings to support prototypes) is cheap and can run on CPU in production once the backbone is pre-trained.

How many meta-training tasks do I need?

This depends on diversity rather than absolute count. A few thousand episodically sampled tasks from a varied class pool is a common baseline. What matters most is that the meta-training classes are numerous and diverse enough to generalize to unseen target classes.

Is few-shot learning the same as zero-shot learning?

No. Zero-shot learning requires no labeled examples from novel classes at test time, relying entirely on semantic class descriptors or attributes. Few-shot learning always provides at least one labeled example per novel class in the support set.

Related methods

Sources

  1. Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., & Kavukcuoglu, K. (2016). Matching Networks for One Shot Learning. Advances in Neural Information Processing Systems (NeurIPS), 29.

    Open publication reference
    Copied.
  2. Finn, C., Abbeel, P., & Levine, S. (2017). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. Proceedings of the 34th International Conference on Machine Learning (ICML), PMLR 70:1126–1135.

    Open publication reference
    Copied.