Skip to content
Back to search results

Text Classification — Text Categorization

Type
Supervised NLP classification task
Font size

What does it do?

Text classification, also called text categorization, is a supervised natural-language-processing task that automatically assigns documents to predefined categories.

Building on the support-vector-machine approach to text categorization established by Joachims (1998) and consolidated in the text-mining literature by Aggarwal and Zhai (2012), it powers tasks such as spam detection and topic classification by learning from labelled examples.

The intuition behind the method

When you have many documents that each belong to one of a fixed set of categories, sorting them by hand does not scale. Text classification learns the patterns that separate one category from another from a set of already-labelled examples, then applies those patterns to label new, unseen documents automatically — much like teaching a sorter by showing it pre-sorted piles and letting it file the rest.

History

Text classification became a benchmark machine-learning problem through Joachims (1998), who showed that support vector machines handle the many relevant features of text categorization well. Aggarwal and Zhai (2012) later consolidated the field within the broader text-mining literature, and the task remains a core building block of applied natural-language processing.

When to use it

Use text classification when you have text data sorted into predefined categories and a labelled training set to learn from. It needs at least roughly 100 labelled documents, and a balanced class distribution is preferred so the classifier can learn each category. With fewer than about 100 documents the classifier tends to overfit, so a zero-shot approach is safer; with no text data at all the method cannot run and general classification on categorical variables is the alternative.

Limitations and considerations

  • Requires labelled training data, which can be costly to produce.
  • A balanced class distribution is preferred; skewed classes make some categories hard to learn.
  • Needs a reasonably sized corpus (around 100 documents or more) to avoid overfitting.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Automates labelling of large document collections that would be impractical to sort by hand.
  • Adapts to many tasks — spam detection, topic classification, and similar categorization problems — through the same supervised pipeline.
  • Learns domain-specific patterns directly from labelled examples rather than relying on fixed rules.

Applications

  • Spam detection — separating unwanted messages from legitimate ones.
  • Topic classification — routing documents or articles into subject categories.
  • Organising and triaging large document collections by category.
  • Prediction and classification tasks across health, social, education, engineering, natural-science, and business text corpora.

How it works

How it works

1

Start from a corpus of documents where each example already carries one of the predefined category labels. This labelled training data is the foundation of the supervised task; the more balanced the class distribution, the more reliably the classifier learns each category.

2

The raw text is cleaned and standardised so that documents become comparable: noise is removed and the content is normalised and segmented into analysable units. Skipping this step lets noisy tokens degrade everything downstream.

3

The cleaned text is turned into numeric features the classifier can read — for example term-frequency representations that capture which words distinguish the categories. This converts words into the numbers the learning algorithm operates on.

4

A supervised classifier is fitted on the labelled, vectorised training data, learning the association between text features and category labels so it can assign a label to each new document.

5

The trained model is scored on held-out labelled data and its misclassifications are inspected before deployment. Because imbalanced classes make raw accuracy misleading, performance is judged on the categories the model is meant to separate.

Common pitfalls

  • Training on too few documents (under ~100) causes overfitting on the small training set; prefer a zero-shot classification approach instead.
  • Skipping text preprocessing lets noisy tokens degrade the features and the model.
  • Imbalanced classes make plain accuracy misleading; inspect performance per category rather than trusting a single accuracy figure.
  • Applying the method when there is no text data at all — fall back to general classification on categorical variables.

Frequently asked questions

How much labelled data do I need?

Plan for at least roughly 100 labelled documents. Below that the classifier tends to overfit the small training set, so a zero-shot classification approach is the safer choice.

Does class balance matter?

Yes. A balanced class distribution is preferred so the classifier can learn each category reliably. With skewed classes, judge performance per category rather than relying on overall accuracy.

What if I have no text data?

Text classification cannot run without text. If your variables are categorical instead, use general classification methods on that structured data.

How is it different from document clustering?

Text classification is supervised: it assigns documents to predefined categories learned from labelled examples. Document clustering is unsupervised and groups similar documents without predefined labels.

Related methods

Sources

  1. Joachims, T. (1998). Text Categorization with Support Vector Machines: Learning with Many Relevant Features. ECML 1998. Lecture Notes in Computer Science, vol 1398. Springer.

    Open DOI reference
    Copied.
  2. Aggarwal, C. C. & Zhai, C. (2012). Mining Text Data. Springer.

    ISBN: 978-1-4614-3222-7
    Copied.