Skip to content
Back to search results

Keyness Analysis

Introduced by
Mike Scott
Year
1997
Type
Corpus comparison of relative word frequencies
Family
Corpus linguistics
Font size

What does it do?

Keyness analysis identifies the words that are characteristically frequent (or infrequent) in a target corpus relative to a reference corpus, using statistical tests to measure how unexpected each word's frequency is.

Introduced by Mike Scott in 1997, it answers the question 'what is this text or collection distinctively about?' and is a central technique in corpus linguistics and corpus-assisted discourse analysis for surfacing the salient vocabulary of a genre, period, author, or social group.

The intuition behind the method

Every text uses common words like 'the' and 'of' at roughly the same rate, so frequency alone does not tell you what makes a text special. Keyness analysis compares a text against a large 'normal' reference and flags the words that stand out — words that occur far more (or far less) often than you would expect by chance. The keywords that emerge are a fingerprint of the text's aboutness: in a corpus of climate reporting they might be 'emissions', 'warming', 'carbon', pointing the analyst straight to what the corpus foregrounds.

History

Mike Scott introduced computerized keyword analysis in his WordSmith Tools software and formalized the notion of key words and 'key key words' in a 1997 System article, drawing on the log-likelihood statistic popularized for corpus comparison by Ted Dunning. Paul Baker's 2006 work established keyness as a cornerstone of corpus-assisted discourse analysis, using it to provide a less subjective entry into large datasets. Costas Gabrielatos and others later refined the methodology, distinguishing significance from effect size and proposing metrics such as %DIFF and log ratio to rank keywords by magnitude.

When to use it

Use keyness analysis when you want a data-driven answer to what is distinctive about a text or corpus compared with a norm — to characterize a genre, track a topic across time, compare how different sources cover an issue, or find a starting point for discourse analysis. It requires a well-chosen reference corpus and is most informative for content words. It is less suitable when no appropriate reference exists, when the corpora are too small for stable frequency estimates, or when the question concerns how words are used rather than which words are salient.

Limitations and considerations

  • Results depend heavily on the reference corpus chosen; a different benchmark yields a different keyword list.
  • With large corpora, significance tests flag trivially small differences as significant unless effect sizes are reported.
  • Keyness is computed per word type, so it can fragment multiword expressions and miss patterns above the single-word level.
  • Keywords indicate salience, not meaning; interpretation still requires returning to context via concordances.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Provides an empirical, replicable way to identify the salient vocabulary of a corpus, reducing analyst cherry-picking.
  • Surfaces both expected and unexpected keywords, often pointing to themes the analyst would not have anticipated.
  • Scales to very large corpora and integrates naturally with concordance analysis for follow-up interpretation.
  • Supports comparison across time, source, or social group, making it powerful for discourse and diachronic studies.

Applications

  • Characterizing the distinctive vocabulary of a genre, register, or specialized domain.
  • Comparing how different newspapers or political actors lexically frame the same issue.
  • Tracking changing salience of topics and terms across time periods in diachronic corpora.
  • Authorship and stylistic studies that compare an author's distinctive word use against a norm.
  • Providing a principled starting list of terms for downstream concordance and collocation analysis in discourse studies.

How it works

How it works

1

Word-frequency lists are generated for two corpora: the target (study) corpus whose distinctive vocabulary is of interest, and a reference (benchmark) corpus representing a relevant norm — often a large, balanced general corpus or a contrasting subcorpus. The choice and size of the reference corpus strongly shape which words emerge as key, so it is selected deliberately.

2

For each word, its observed frequency in the target is compared against the frequency expected given the reference, using a statistical test. The most common is the log-likelihood ratio (G²), which sums observed counts times the log of observed-over-expected across the contingency cells; chi-squared is an older alternative. The statistic measures how surprising the word's frequency is under the assumption that the two corpora draw on the same distribution.

Formula
G^2 = 2\sum_i O_i \ln\!\frac{O_i}{E_i}
3

Words are ranked by their keyness scores, with those exceeding a significance threshold flagged as positive keywords (over-represented in the target) or negative keywords (under-represented). Because large corpora make even tiny differences statistically significant, good practice supplements the significance test with an effect-size measure such as %DIFF or log ratio, as Gabrielatos emphasizes, to rank by magnitude rather than mere significance.

4

The resulting keyword list is interpreted, typically by grouping keywords thematically and then returning to concordance lines to see how each keyword is actually used. In corpus-assisted discourse analysis, as Baker shows, keywords provide an empirical, less cherry-picked entry point into a corpus, pointing the analyst toward the lexical patterns through which discourses are constructed.

Common pitfalls

  • Relying on statistical significance alone in large corpora, producing long lists dominated by trivially over-represented function words.
  • Choosing a reference corpus that is mismatched in genre or era, which contaminates the keyword list with artifacts of that mismatch.
  • Interpreting keywords without consulting concordances, leading to misreadings of polysemous or ambiguous items.
  • Comparing raw counts instead of normalized frequencies when the two corpora differ greatly in size.

Frequently asked questions

What statistic is used to compute keyness?

The most common is the log-likelihood ratio (G²), which compares observed and expected frequencies across the two corpora and is robust to the skewed, low-count distributions typical of word frequencies. Chi-squared was used historically but is less reliable for rare items. Increasingly, analysts report log-likelihood for significance alongside an effect-size metric such as %DIFF or log ratio to rank keywords by how large, not just how reliable, the difference is.

How do I choose a reference corpus?

The reference corpus defines 'normal', so it should be larger than the target and appropriate to the comparison. A general balanced corpus (such as a national reference corpus) suits questions about what is distinctive relative to language at large, while a contrasting subcorpus suits questions about what differs between two specific varieties. A mismatched reference produces keywords that merely reflect the mismatch, so the choice must be justified.

Why report effect size as well as significance?

In large corpora, the significance test has so much power that even negligible frequency differences become 'significant', producing inflated keyword lists dominated by high-frequency function words. Effect-size measures such as %DIFF or log ratio quantify how much more frequent a word is, allowing analysts to rank by substantive importance rather than statistical detectability, which Gabrielatos and others argue is essential for meaningful keyness analysis.

Related methods

Sources

  1. Scott, M. (1997). PC analysis of key words — and key key words. System, 25(2), 233–245.

    Open DOI reference
    Copied.
  2. Baker, P. (2006). Using Corpora in Discourse Analysis. Continuum.

    ISBN: 9780826477248
    Copied.
  3. Gabrielatos, C. (2018). Keyness analysis: Nature, metrics and techniques. In C. Taylor & A. Marchi (Eds.), Corpus Approaches to Discourse: A Critical Review (pp. 225–258). Routledge.

    ISBN: 9781138895157
    Copied.