Skip to content
Back to search results

Collocation Analysis — Word Association

Introduced by
Church & Hanks
Year
1990
Type
Statistical text-mining technique
Font size

What does it do?

Collocation analysis is a statistical text-mining technique that identifies word pairs or expressions that frequently occur together, using association measures rather than chance co-occurrence.

Introduced in the lexicography work of Church and Hanks (1990), it is used for terminology extraction and language analysis, surfacing the multi-word units that carry meaning in a corpus.

The intuition behind the method

Some words simply belong together — 'strong tea' sounds natural while 'powerful tea' does not, even though the meaning is similar. Collocation analysis looks past raw frequency and asks which pairs of words appear together far more often than chance would predict. By scoring co-occurrence statistically, it pulls out the meaningful phrases and fixed expressions hidden inside a body of text instead of just listing common words.

History

Collocation analysis as a statistical technique was established by Church and Hanks (1990), who introduced word-association norms and mutual information for lexicography, showing how co-occurrence statistics could surface meaningful word pairs. The approach was consolidated within the broader field of statistical natural-language processing by Manning and Schütze (1999).

When to use it

Collocation analysis fits when you have a reasonably sized text corpus (around 50 or more documents) and want to find meaningful multi-word units rather than single-word frequencies. It suits exploratory and descriptive work in terminology extraction and linguistic analysis. It assumes enough text for stable co-occurrence statistics and that stop-words have been cleaned first; with very little text the association scores become unreliable.

Limitations and considerations

  • Needs a sufficiently large corpus for the co-occurrence statistics to be stable.
  • Results depend heavily on preprocessing — leftover stop-words distort the associations.
  • Measures like PMI can over-reward rare word pairs that happen to co-occur a few times.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Surfaces meaningful multi-word expressions and fixed phrases that single-word frequency counts miss.
  • Statistical association measures separate genuine word associations from chance adjacency.
  • Conceptually simple and easy to apply for terminology extraction and language analysis.

Applications

  • Terminology extraction — identifying domain-specific multi-word terms in a corpus.
  • Lexicography and dictionary building, surfacing characteristic word associations.
  • Linguistic and discourse analysis of how words combine in a body of text.
  • Preparing candidate phrases for downstream text-mining and information-extraction tasks.

How it works

How it works

1

The corpus is tokenised and cleaned, with stop-words removed so that high-frequency function words do not drown out meaningful associations. Consistent preprocessing is what keeps the co-occurrence counts comparable and the resulting collocations interpretable.

2

Candidate word pairs (typically bigrams) are formed and their co-occurrence counts are gathered across the corpus, alongside the individual frequencies of each word. These counts are the raw material every association measure is built from.

3

Each candidate pair is scored with a statistical association measure — such as pointwise mutual information (PMI) — that compares how often the words appear together against how often they would be expected to co-occur by chance. A high score flags a genuine association rather than an accidental adjacency.

4

Candidate collocations are ranked by their association scores and the strongest pairs are selected (for example, the top-N by PMI). The result is a ranked list of meaningful word pairs and multi-word expressions for terminology extraction or language analysis.

Common pitfalls

  • Skipping stop-word removal lets high-frequency function words dominate and bury real collocations.
  • Running the analysis on too little text yields unstable, unreliable association scores.
  • PMI inflates the scores of very rare pairs, so apply a minimum co-occurrence count before ranking.
  • Treating a high-frequency pair as a collocation without an association test mistakes commonness for meaningful association.

Frequently asked questions

What exactly is a collocation?

A collocation is a pair or sequence of words that co-occurs more often than chance would predict — a habitual word association such as 'strong tea' or 'make a decision'. Collocation analysis finds these by scoring co-occurrence statistically rather than reading them off a frequency list.

How is collocation analysis different from frequency analysis?

Frequency analysis counts how often words or n-grams appear. Collocation analysis goes further: it asks whether two words appear together more than expected by chance, using an association measure, so it surfaces meaningful pairings rather than merely common ones.

What is PMI and why is it used?

Pointwise mutual information (PMI) compares how often two words occur together against how often they would be expected to co-occur independently. A high PMI flags a strong association. Because PMI can over-reward rare pairs, a minimum co-occurrence count is usually applied before ranking.

How much text do I need?

The technique expects a reasonably sized corpus — on the order of 50 or more documents — so that co-occurrence counts are stable. With too little text the association scores fluctuate and the extracted collocations become unreliable.

Related methods

Sources

  1. Church, K.W. & Hanks, P. (1990). Word Association Norms, Mutual Information, and Lexicography. Computational Linguistics, 16(1), 22-29.

    Open publication reference
    Copied.
  2. Manning, C.D. & Schütze, H. (1999). Foundations of Statistical Natural Language Processing. MIT Press.

    ISBN: 9780262133609
    Copied.