Chunking — Shallow Parsing
- Introduced by
- Steven Abney
- Year
- 1991
- Type
- NLP partial-parsing task
What does it do?
Chunking, also called shallow parsing, is a natural-language-processing task introduced by Steven Abney in 1991 that divides text into grammatical pieces — such as noun phrases and verb phrases — using part-of-speech tags.
It extracts useful syntactic structure quickly without building a full parse tree of the sentence.
The intuition behind the method
A full syntactic parse describes how every word relates to every other word, which is expensive and often more than an application needs. Chunking takes a shortcut: starting from words already tagged with their parts of speech, it groups adjacent tags into flat, non-overlapping phrases. You learn that 'the small dog' is a noun phrase and 'is barking' is a verb phrase without resolving the complete grammatical tree, giving fast structural insight at low cost.
History
Chunking was introduced by Steven Abney (1991) in 'Parsing by Chunks', which framed partial parsing around chunks as an efficient alternative to full syntactic analysis. The task was later standardised and popularised by the CoNLL-2000 shared task on chunking (Tjong Kim Sang & Buchholz, 2000), which established a common dataset and evaluation for the problem.
When to use it
Use chunking when you have part-of-speech tagged text and need phrase-level structure quickly without the cost of a full syntactic parse. It is suited to exploratory and descriptive analysis of text and needs only a modest amount of data (around ten units). Two conditions must hold: POS tagging must already be complete, and a chunk grammar or model must exist for the target language.
Limitations and considerations
- Requires part-of-speech tagging to be completed first; errors in the tags propagate into the chunks.
- Needs a chunk grammar or model that matches the target language.
- Produces only flat, non-overlapping phrases — it does not recover the full nested syntactic structure.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Extracts useful syntactic structure fast, without the cost of building a full parse tree.
- Works from POS tags with simple, transparent pattern rules that are easy to inspect and adjust.
- Provides phrase-level building blocks (noun and verb phrases) for downstream tasks such as information extraction.
Applications
- Information extraction — isolating noun and verb phrases as candidate entities and relations.
- Preprocessing for named-entity recognition by surfacing noun-phrase boundaries.
- Fast structural analysis of large text corpora where full parsing is too costly.
- Exploratory and descriptive study of grammatical phrase patterns in a corpus.
How it works
How it works
The method starts from text that has already been part-of-speech tagged — each token carries a label such as determiner, adjective, noun, or verb. Completed POS tagging is a prerequisite, because the chunker works on these tags rather than on raw words.
A chunk grammar is specified for the target language, typically as patterns over POS tags (for example, a noun phrase as an optional determiner, any adjectives, then a noun). A rule set or trained model for the language must be available for this to succeed.
The grammar is applied across the tagged sequence, grouping adjacent tags that match a pattern into flat, non-overlapping chunks. Unlike full parsing, chunks are not nested into a complete tree — the parse stays shallow.
The identified phrase chunks — noun phrases, verb phrases, and the like — are read off as the output. These shallow structures feed downstream tasks such as information extraction or named-entity recognition.
Common pitfalls
- Running chunking on untagged or poorly tagged text — POS tagging must be done first, and tagging errors carry straight into the chunks.
- Reusing a chunk grammar built for one language on another; the pattern rules must match the target language.
- Expecting nested or complete syntactic structure from a shallow parser — chunks are flat by design, so use full parsing when deep structure is required.
Frequently asked questions
How is chunking different from full parsing?
Full parsing builds a complete syntactic tree showing how every word relates to every other word. Chunking is a shallow parse: it groups adjacent words into flat, non-overlapping phrases such as noun and verb phrases, without nesting them into a full tree. This makes it much faster and often sufficient for tasks that only need phrase-level structure.
Do I need part-of-speech tagging before chunking?
Yes. Chunking operates on POS tags rather than raw words, so tagging must be completed first. Because the chunker relies on those tags, any errors in tagging carry directly into the chunks.
Does chunking work for any language?
It works only where a chunk grammar or trained model exists for the target language. The pattern rules are language-specific, so a grammar written for one language should not be applied to another.
What does chunking produce?
It produces flat, non-overlapping phrase chunks — typically noun phrases and verb phrases — read off the tagged sequence. These shallow structures are commonly used as input to information extraction and named-entity recognition.
Related methods
Sources
Abney, S. (1991). Parsing by Chunks. In Principle-Based Parsing. Kluwer Academic Publishers.
ISBN: 978-0-7923-1173-4Copied.Tjong Kim Sang, E.F. & Buchholz, S. (2000). Introduction to the CoNLL-2000 Shared Task: Chunking. Proceedings of CoNLL-2000.
Open publication referenceCopied.