Skip to content
Back to search results

Sentiment Analysis — Opinion Mining

Type
NLP text-classification task
Font size

What does it do?

Sentiment analysis, also called opinion mining, is a natural-language-processing task that detects the emotional tone of text — typically classifying it as positive, negative, or neutral. It turns unstructured opinion text into structured, quantifiable polarity signals using one of three families of approaches: sentiment lexicons, trained machine-learning classifiers, or pretrained transformer models.

The intuition behind the method

People express opinions in free-form language, but a machine cannot read that text the way a human does. The goal of sentiment analysis is to detect the underlying opinion or polarity in a piece of text and attach a label to it. Instead of judging each sentence by intuition, the method scores wording — through dictionaries of charged words, statistical patterns learned from labelled examples, or context captured by a language model — and assigns each document a sentiment label.

History

Sentiment analysis grew out of the broader field of opinion mining surveyed by Pang and Lee (2008), who consolidated the task of extracting opinion and polarity from text. The field has since moved from dictionary- and feature-based methods toward machine-learning classifiers and, more recently, pretrained transformer models that capture context.

When to use it

Sentiment analysis fits when you have text data to work with and a language model or lexicon that matches the language of that text. A small corpus can be handled with a simple lexicon, while larger labelled corpora support supervised classifiers or transformers. Supervised models in particular need a reasonably large, balanced labelled set; with very few documents a lexicon-based method is the safer choice. If there is no text data at all, sentiment analysis cannot run.

Limitations and considerations

  • Supervised models need a sufficiently large, balanced labelled corpus to learn reliable patterns.
  • Performance depends on the language model or lexicon matching the corpus language.
  • Sarcasm, negation, and domain shift remain hard for every approach.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Flexible across approaches: a simple lexicon works on small corpora, while transformers capture context in large ones.
  • Turns unstructured opinion text into structured, quantifiable polarity signals.
  • Scales from quick dictionary scoring to context-aware language models without changing the overall pipeline.

Applications

  • Analysing product and service reviews to summarise customer opinion.
  • Monitoring social media posts for public reaction and sentiment trends.
  • Brand monitoring — tracking how an organisation or product is perceived over time.
  • Exploring and classifying large opinion corpora in operations and marketing research.

How it works

How it works

1

The raw corpus is cleaned and standardised: text is lowercased, noise is stripped, content is normalised, and documents are segmented (tokenised) into analysable units. Consistent preprocessing is what keeps the downstream features comparable across documents.

2

The cleaned text is converted into features a model can consume. Depending on the approach this means sentiment-lexicon lookups, TF-IDF vectors, or contextual embeddings — each turning words into numbers that carry the signal the model will read.

3

A polarity label is assigned to each document. A lexicon-based approach applies dictionary rules, a machine-learning approach uses a classifier trained on labelled data, and a transformer-based approach applies a pretrained language model that captures context. The output is a sentiment label per document.

4

Predictions are scored against labelled data using accuracy, F1, precision, and recall, and misclassifications are inspected before the model is put to use. This step guards against over-trusting a model that performs poorly on certain classes.

Common pitfalls

  • Sarcasm and figurative language can invert the apparent polarity, so a literal reading mislabels the text.
  • Domain shift — applying a model or lexicon built for one domain or language to another — degrades accuracy; do not apply an English lexicon to text in another language.
  • Imbalanced labels make accuracy misleading; report F1, precision, and recall rather than accuracy alone.
  • Running a supervised classifier on too few documents (under ~100) — fall back to a lexicon-based method, and never skip preprocessing or noisy tokens will degrade the features.

Frequently asked questions

Lexicon, machine learning, or transformer — which approach should I use?

A lexicon-based approach applies a dictionary of sentiment-bearing words and works well on small corpora with no labelled data. A machine-learning classifier learns patterns from a labelled training set and adapts to your domain. A transformer-based model uses a pretrained language model to capture context and generally performs best on large corpora, at higher computational cost.

How accurate is sentiment analysis?

Accuracy varies with the approach, the corpus, and how well the model matches the text. Evaluate with accuracy, F1, precision, and recall on held-out labelled data; on imbalanced classes, accuracy alone can be misleading, so rely on F1, precision, and recall.

Can it work in languages other than English?

Yes, but the chosen language model or lexicon must match the target language of the corpus. Applying an English lexicon or model to text in another language is a common cause of poor results.

How are negation and sarcasm handled?

These remain among the hardest cases. Negation can flip polarity and sarcasm can invert the literal meaning entirely. Context-aware transformer models cope better than simple lexicons, but no approach handles sarcasm reliably, so error inspection during evaluation matters.

Related methods

Sources

  1. Pang, B. & Lee, L. (2008). Opinion Mining and Sentiment Analysis. Foundations and Trends in Information Retrieval, 2(1-2), 1-135.

    Open DOI reference
    Copied.