Contribution Analysis
- Introduced by
- John Mayne
- Year
- 2001
- Type
- Theory-based approach to causal inference about contribution
- Family
- Program evaluation methodology
What does it do?
Contribution analysis is a theory-based evaluation approach that addresses the attribution problem — establishing whether and how an intervention made a difference — without relying on an experimental counterfactual. Developed by John Mayne from 2001 onward, it works by articulating the program's theory of change, gathering evidence along that chain, and then assembling a 'contribution story' that is progressively stress-tested against rival explanations.
The aim is not statistical attribution but a credible, evidence-based conclusion that the program plausibly contributed to observed results, in the face of other influencing factors.
The intuition behind the method
Most policy outcomes are over-determined: many things were happening at once, so it is rarely possible to run a clean controlled experiment and claim the program alone caused the result. Contribution analysis treats causal claims like a courtroom argument. You set out the story of how the program was supposed to lead to the outcome, present the evidence that each link in the chain actually held, then deliberately hunt for other explanations — the economy improved anyway, a parallel program ran, the population changed. If the story survives that cross-examination and the rival explanations can be discounted or shown to be minor, you can reasonably conclude the program made a meaningful contribution. The verdict is a reasoned, evidence-weighted judgment, not a p-value.
History
John Mayne, a Canadian evaluation specialist, introduced contribution analysis in a 2001 paper in the Canadian Journal of Program Evaluation as a way to use performance-monitoring data 'sensibly' to address attribution. The approach responded to the dominance of experimental impact evaluation, which was often impossible for the complex public-sector programs evaluators actually faced. Mayne refined the six-step process over the following decade, and his 2012 article 'Contribution analysis: Coming of age?' in Evaluation consolidated it and connected it to philosophical work on contributory causes. It has since become a mainstay of theory-based evaluation in development agencies and the public sector.
When to use it
Use contribution analysis when a counterfactual experiment is impossible, unethical or uninformative — typically for complex, multi-actor policies and programs influenced by many external factors — yet decision-makers still need a credible verdict on whether the intervention made a difference. It is especially valuable when reasonable program monitoring data already exist and a plausible theory of change can be articulated. It is less suited where a clean controlled comparison is feasible and a precise effect size is required, or where no coherent program theory or supporting evidence is available. It pairs naturally with theory of change and realist evaluation as part of the theory-based evaluation family.
Limitations and considerations
- Yields a reasoned, qualitative judgment of contribution rather than a quantified effect size, which some audiences find less authoritative.
- Depends heavily on the quality of the theory of change and the available evidence; weak inputs produce a weak and unconvincing story.
- Rigour in testing alternative explanations is hard to standardise, so the strength of conclusions varies with the evaluator's diligence and skill.
- Can drift toward confirmation of the official program story if rival explanations are sought only perfunctorily.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Provides a structured, defensible way to make causal claims where experimental counterfactuals are infeasible, which covers most real-world policy.
- Forces explicit attention to alternative explanations and external factors rather than assuming the program was the sole cause.
- Makes economical use of existing monitoring and performance data, lowering cost relative to bespoke impact evaluations.
- Produces a transparent narrative whose logic and evidence stakeholders can scrutinise, increasing the credibility and uptake of conclusions.
Applications
- Evaluating international development programs where many donors and contextual factors jointly influence outcomes.
- Assessing the contribution of public-sector policy reforms whose effects unfold over years amid changing conditions.
- Demonstrating results for accountability and performance reporting when randomised designs are infeasible.
- Evaluating advocacy, capacity-building and systems-change initiatives whose impacts are diffuse and indirect.
- Synthesising existing monitoring and evaluation data into a credible causal narrative for funders and policymakers.
How it works
How it works
The evaluator frames the cause-and-effect question precisely: what change is being claimed, for whom, and over what period, and why attribution is contested. This step also acknowledges the other factors that could plausibly explain the outcome and sets the level of confidence that decision-makers actually need, since a definitive causal claim is rarely required or attainable.
A theory of change is constructed setting out the causal chain from the program's activities through outputs to intermediate and final outcomes, together with the assumptions and risks at each link. Crucially, the theory makes explicit the external factors expected to influence results, so they can later be examined rather than ignored.
Evidence is assembled — often from existing monitoring data, prior studies and stakeholder accounts — to assess each link in the chain. The question at every link is whether the expected result occurred and whether the assumptions held, building up a body of evidence that supports or weakens the postulated pathway.
The evidence is woven into a contribution story: a coherent narrative stating that the program was implemented as planned, the chain of intermediate results materialised, the underlying assumptions held, and other influencing factors have been recognised. The story makes the causal claim explicit and exposes its weakest points.
The draft story is challenged by seeking out alternative explanations and gaps in evidence, then gathering additional data to fill them. Where rival explanations cannot be discounted, the contribution claim is qualified or revised. Iterating this loop strengthens the story until a defensible, appropriately hedged conclusion about the program's contribution can be reached.
Common pitfalls
- Building the contribution story but never seriously searching for alternative explanations, which reduces the method to advocacy for the program.
- Confusing a plausible theory of change with evidence that the chain actually held — each link must be verified, not assumed.
- Over-claiming a definitive causal conclusion when the appropriate output is a credible, qualified statement of contribution.
- Ignoring documented external factors that the theory of change itself flagged, leaving an obvious gap a critic can exploit.
Frequently asked questions
Does contribution analysis prove that a program caused an outcome?
No — and it does not claim to. Its goal is a credible, evidence-based conclusion that the program plausibly contributed to the outcome, given the other factors at work. The standard is reasoned confidence rather than the definitive attribution an experiment seeks. The strength of that conclusion rests on how well the theory of change holds up, how complete the evidence is, and how thoroughly alternative explanations have been examined and discounted.
How is contribution analysis different from a randomised controlled trial?
An RCT estimates a counterfactual by randomly assigning a control group and reports a quantified effect size with statistical uncertainty. Contribution analysis is used precisely when such a counterfactual is unavailable; it builds and tests a causal narrative against rival explanations instead of constructing a control group. The two answer related questions with very different evidence: an RCT asks how much, on average; contribution analysis asks whether and how the program contributed amid everything else.
What is a 'contribution story'?
A contribution story is the structured narrative at the heart of the method. It states that the program was implemented as intended, that the expected chain of intermediate results occurred, that the assumptions underpinning each link held, and that other influencing factors have been identified and weighed. A robust contribution story has survived deliberate attempts to find alternative explanations and gaps, and it openly flags any remaining uncertainty in the causal claim.
Related methods
Sources
Mayne, J. (2012). Contribution analysis: Coming of age? Evaluation, 18(3), 270–280.
Open DOI referenceCopied.Mayne, J. (2001). Addressing attribution through contribution analysis: Using performance measures sensibly. Canadian Journal of Program Evaluation, 16(1), 1–24.
Open DOI referenceCopied.