Few-Shot Text Classification
- Type
- NLP text-classification task (low-resource)
What does it do?
Few-shot text classification assigns documents to classes using only a handful of labelled examples per class.
Building on advances by Gao et al. (2021) and the prompt-free SetFit approach of Tunstall et al. (2022), it leans on prototypical networks, MAML, or fine-tuning of a large pretrained model to learn from scarce labels.
The intuition behind the method
Most classifiers need a large labelled training set, but labelling text is expensive and sometimes only a few examples per class exist. Few-shot text classification works around this by starting from a model that already understands language — a large pretrained model — and nudging it toward your classes with just a few examples each. Instead of learning every class from scratch, it compares new documents to a small set of labelled prototypes or lightly adapts the pretrained model so even one example per class can guide the decision.
History
Few-shot text classification emerged from few-shot and meta-learning research adapted to language. Gao et al. (2021) showed how to make pretrained language models better few-shot learners, and Tunstall et al. (2022) introduced SetFit for efficient few-shot learning without prompts, making the approach practical with small labelled sets and strong pretrained backbones.
When to use it
Use few-shot text classification when you have text to classify but only a few labelled examples per class — far fewer than a conventional supervised classifier would need. It assumes at least one example exists for every class and works best when a large pretrained model is available to supply the missing knowledge. With abundant labelled data, an ordinary supervised text classifier is the simpler choice.
Limitations and considerations
- Needs at least one labelled example for every class; entirely unseen classes cannot be predicted.
- Quality depends heavily on the pretrained backbone — a weak or mismatched model limits performance.
- With extremely few examples, predictions can be unstable and sensitive to which examples were chosen.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Learns from very few labelled examples — as little as one per class — where conventional supervised classifiers fail.
- Reuses the broad language knowledge inside a large pretrained model, so little task-specific data is needed.
- Flexible across strategies: prototypical networks, MAML, or lightweight fine-tuning all fit the same pipeline.
Applications
- Classifying text when labelling is costly and only a few examples per category are available.
- Bootstrapping a classifier for a new domain or label set before large-scale labelling is done.
- Rapid prototyping of text categorisation in research and operational settings with scarce labels.
- Extending an existing scheme with new, sparsely-labelled classes.
How it works
How it works
You assemble a small labelled set with at least one example per class — sometimes only a few per class. This is the support set the method learns from; the smaller it is, the more the method relies on knowledge already inside the pretrained model.
A large pretrained model is chosen as the backbone, because it already encodes broad language knowledge. A strong pretrained model is preferred — the more it knows about language up front, the less each class needs to be taught from your few examples.
The backbone is steered toward your classes with one of the few-shot strategies: prototypical networks build a class prototype from the few examples and classify by nearest prototype, MAML meta-learns initial parameters that adapt quickly, and fine-tuning lightly updates the pretrained model on the small set.
Each new document is encoded by the adapted model and assigned a class label — by nearest prototype or by the fine-tuned classifier head — yielding one predicted label per document despite the scarce training data.
Common pitfalls
- Treating a class with zero labelled examples as learnable — every class must have at least one example.
- Pairing the method with a small or poorly matched pretrained model, which removes the prior knowledge the few-shot setting depends on.
- Over-reading accuracy from a tiny support set; with so few examples, evaluation estimates are noisy.
Frequently asked questions
How few examples per class does it really need?
At least one labelled example per class is required, and the method is designed to work with only a few per class. The fewer you provide, the more it leans on the pretrained backbone's prior knowledge.
Which few-shot strategy should I use?
Prototypical networks build a prototype per class from the few examples and classify by nearest prototype; MAML meta-learns parameters that adapt quickly to new tasks; fine-tuning lightly updates a pretrained model on the small set. All three fit the same pipeline; the prompt-free SetFit style of fine-tuning is a practical default.
Why does the pretrained model matter so much?
With only a few labelled examples, most of the knowledge has to come from somewhere else. A large pretrained model supplies broad language understanding, so a stronger backbone generally gives better few-shot results.
When should I prefer a normal supervised classifier instead?
When you have abundant labelled data. Few-shot methods exist for the low-label regime; with a large balanced labelled set, a standard supervised text classifier is simpler and usually stronger.
Related methods
Sources
Gao, T., Fisch, A. & Chen, D. (2021). Making Pre-trained Language Models Better Few-shot Learners. ACL.
Open DOI referenceCopied.Tunstall, L., Reimers, N., Jo, U.E.S., Bates, L., Korat, D., Wasserblat, M. & Pereg, O. (2022). Efficient Few-Shot Learning Without Prompts. arXiv.
Open DOI referenceCopied.