Question Answering (QA)
- Type
- NLP text-comprehension task
What does it do?
Question answering is a natural-language-processing task that automatically answers natural-language questions grounded in a given context passage, using either extractive or generative approaches.
The task was crystallised by the SQuAD benchmark of Rajpurkar et al. (2016), and later models such as XLNet (Yang et al., 2019) pushed reading-comprehension accuracy higher.
The intuition behind the method
When you ask a question, the answer usually lives somewhere inside a relevant body of text. A question-answering system reads that context passage alongside the question and works out the response. An extractive system locates and copies the exact span of text that answers the question, while a generative system writes an answer in its own words. Either way, the model is anchored to the supplied context rather than guessing freely.
History
Modern question answering was reshaped by the release of SQuAD (Rajpurkar et al., 2016), a large reading-comprehension dataset of over 100,000 questions posed against context passages, which gave the field a common benchmark for machine comprehension. Subsequent pretraining methods such as XLNet (Yang et al., 2019) advanced the autoregressive language modelling behind the strongest QA systems.
When to use it
Question answering fits when you have text questions to answer and a context passage that is expected to contain those answers. It suits exploratory and descriptive work over textual data, and benefits from a modest set of examples to test against. A domain-specific model is worth using when the text is specialised. Without a context passage the method has nothing to read, and a general model on niche terminology will underperform.
Limitations and considerations
- Requires a relevant context passage; with no context to read, the method cannot answer reliably.
- A general-purpose model gives lower accuracy on domain-specific text than a model tuned for that domain.
- Quality depends on how well the answer is actually contained in, and phrased within, the provided passage.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Grounds answers in a supplied context passage rather than free-floating generation, keeping responses traceable to source text.
- Supports both extractive answers — exact spans copied from the text — and generative answers written in natural language.
- Domain-specific models can be slotted in to raise accuracy on specialised material without changing the overall pipeline.
Applications
- Answering user questions against a document or knowledge base.
- Reading-comprehension systems that extract precise answer spans from passages.
- Customer- and research-support tools that respond to natural-language queries over a text corpus.
- Information access over technical or domain-specific documents using a tuned model.
How it works
How it works
The system takes two inputs: a natural-language question and a context passage assumed to contain the answer. Because the method is grounded in the supplied context, having a relevant passage available is a prerequisite for any sensible answer.
The question and context are read together by a language model that builds a joint representation, capturing how the words of the question relate to the words of the passage. A domain-specific model encodes terminology more accurately and yields higher accuracy on specialised text.
An answer is produced. An extractive approach predicts the start and end of the span in the context that best answers the question and returns that exact text; a generative approach writes a free-form answer conditioned on the question and context. The output is a natural-language answer tied to the passage.
Predicted answers are scored against reference answers — typically with exact-match and F1 measures on a benchmark such as SQuAD — and failure cases are inspected before the system is deployed.
Common pitfalls
- Supplying a context passage that does not actually contain the answer forces the model to guess, producing confident but wrong responses.
- Applying a general model to highly specialised terminology degrades accuracy; prefer a domain-specific model for niche text.
- Judging extractive and generative outputs by the same yardstick can mislead — extractive answers are best scored with exact-match and F1 against the reference span.
- Skipping evaluation hides systematic failure modes; inspect wrong answers on a benchmark before trusting the system.
Frequently asked questions
What is the difference between extractive and generative question answering?
An extractive approach finds and returns the exact span of text in the context that answers the question. A generative approach writes an answer in its own words, conditioned on the question and context. Extractive answers stay verbatim and are easy to score against the source; generative answers read more naturally but can drift from the passage.
Do I always need a context passage?
In this method, yes. The system answers based on a supplied context that is assumed to contain the answer. Without a relevant passage to read, the model has nothing to ground its answer in and reliability drops sharply.
Should I use a domain-specific model?
If your text is specialised, a domain-specific model gives higher accuracy than a general-purpose one, because it encodes the terminology more faithfully. For general text, a general model is usually adequate.
How is question-answering accuracy measured?
Predicted answers are compared with reference answers using measures such as exact match and F1, typically on a benchmark like SQuAD. Inspecting failure cases alongside these scores reveals systematic weaknesses before deployment.
Related methods
Sources
Rajpurkar, P. et al. (2016). SQuAD: 100,000+ Questions for Machine Comprehension of Text. EMNLP.
Open DOI referenceCopied.Yang, Z. et al. (2019). XLNet: Generalized Autoregressive Pretraining for Language Understanding. NeurIPS.
Open publication referenceCopied.