Participatory Impact Assessment
- Introduced by
- Andy Catley and colleagues, Feinstein International Center, Tufts University
- Year
- 2014
- Type
- Participatory project impact assessment method
- Family
- Participatory monitoring and evaluation
What does it do?
Participatory Impact Assessment (PIA) is an approach to measuring the impact of development and humanitarian projects in which the affected communities define the indicators of change and use participatory tools to quantify it.
Developed and codified by Andy Catley and colleagues at Tufts University's Feinstein International Center, largely through work on livestock and livelihoods programmes in pastoralist settings, PIA adapts participatory rural appraisal methods to the disciplined logic of impact evaluation — combining locally meaningful indicators with before-and-after and with-and-without comparisons to assess what a project actually changed.
The intuition behind the method
Classic impact evaluation prizes counterfactuals and externally chosen indicators, but in fluid, hard-to-survey settings such as pastoralist drylands a rigorous control group is often impossible and outsider indicators may miss what matters. PIA's insight is that communities can both define meaningful indicators of impact and measure change reliably if given structured participatory tools. By asking people to score, for instance, the relative contribution of different income sources before and after an intervention — using methods like proportional piling where a pile of counters is divided across categories — PIA produces semi-quantitative estimates of impact that are grounded in local reality yet structured enough to compare and triangulate.
History
PIA crystallised from work at Tufts University's Feinstein International Center, where Andy Catley and colleagues sought rigorous yet practical ways to evaluate impact in pastoralist and humanitarian livelihoods programmes — settings where the standard evaluation toolkit struggled. It drew directly on Robert Chambers's participatory rural appraisal and on participatory epidemiology methods developed for community-based animal health. The 'Participatory Impact Assessment: A Design Guide', first issued in 2008 and revised in 2014, codified the eight-stage process and the suite of scoring tools, and PIA has since been applied by NGOs and agencies well beyond its pastoralist origins.
When to use it
Use PIA when you need credible, locally grounded estimates of project impact in contexts where conventional experimental or survey-based evaluation is impractical — remote, mobile, insecure, or rapidly changing populations such as pastoralists, or emergency and livelihoods programmes where no clean control group exists. It is well suited to measuring impact on dimensions communities themselves can judge and where semi-quantitative, triangulated evidence is acceptable. It is less appropriate where a rigorous statistical counterfactual is both feasible and required, where precise monetary valuation is essential, or where results must be nationally representative.
Limitations and considerations
- Estimates are semi-quantitative and site-specific, lacking the statistical precision and external validity of large-sample designs.
- Without a genuine control group, attribution rests on weaker comparisons and is vulnerable to confounding by external trends.
- Results depend on facilitator skill and consistent tool application; poorly run scoring exercises yield unreliable numbers.
- Aggregating proportional-piling and scoring outputs across groups requires care to avoid spurious precision.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Generates semi-quantitative impact estimates in settings where randomised or survey-based counterfactual designs are infeasible.
- Uses community-defined indicators, so impact is measured on dimensions that are locally meaningful and often missed by external metrics.
- Participatory scoring tools such as proportional piling are accessible to non-literate participants and quick to apply in the field.
- Built-in before/after and with/without comparisons plus triangulation strengthen attribution beyond simple description.
Applications
- Assessing the impact of livestock, animal-health, and pastoralist livelihoods programmes in the Horn of Africa.
- Evaluating emergency and resilience interventions where no statistical control group is available.
- Measuring changes in income sources and dietary diversity using proportional piling before and after a project.
- Providing accountability evidence to donors for community-based development projects in remote areas.
- Complementing logframe and theory-of-change evaluations with community-grounded indicators of change.
How it works
How it works
The assessment starts from clear impact questions linked to the project's logic: what change was the intervention expected to produce, for whom, and through what pathway? Framing precise, answerable questions — and being explicit about the project's theory of change — keeps the participatory work focused on attributable impact rather than open-ended description, and determines which comparisons (before/after, with/without) are needed.
Working with community members, the team identifies locally meaningful indicators of the changes in question — for example shifts in herd size, milk availability, dietary diversity, income sources, or coping capacity. Using community-defined indicators ensures that what is measured is what people actually value and can judge, and surfaces dimensions of impact that externally fixed indicators would overlook.
Impact is quantified using structured participatory tools, principally proportional piling (dividing a fixed set of counters across categories to show relative magnitude), before-and-after scoring, matrix scoring, and simple ranking. Conducted with multiple separate groups, these tools generate semi-quantitative, comparable data on the direction and size of change while remaining accessible to non-literate participants.
Scores are repeated across several informant groups and cross-checked against key-informant interviews, direct observation, and any secondary or conventional data, so that findings are triangulated rather than resting on a single exercise. Attribution is strengthened by built-in comparisons — before versus after the intervention and, where feasible, with versus without project participation — and by probing for alternative explanations of the observed change.
Common pitfalls
- Skipping the project's theory of change, so the exercise measures activity or output rather than attributable impact.
- Imposing outsider indicators instead of eliciting community-defined ones, which undermines the method's core rationale.
- Running a single group exercise and treating its scores as definitive rather than repeating across groups and triangulating.
- Presenting proportional-piling percentages as if they were precise survey statistics, overstating their accuracy.
Frequently asked questions
What is proportional piling and why is it central to PIA?
Proportional piling is a participatory tool in which a group is given a fixed quantity of counters (such as beans or stones) and asked to divide the pile across categories so that the size of each sub-pile reflects its relative importance — for example the share of household income from different sources. Repeated before and after an intervention, it yields semi-quantitative estimates of change that are accessible to non-literate participants, which is why it is a workhorse method in PIA.
How does PIA handle attribution without a control group?
PIA strengthens attribution through design rather than statistics: it builds in before-and-after comparisons and, where possible, with-and-without comparisons between participants and non-participants, repeats exercises across multiple informant groups, and triangulates the scores against interviews, observation, and secondary data. It also explicitly probes for alternative explanations. Attribution is therefore plausibility-based rather than experimentally proven, which is appropriate for the settings PIA targets.
How is PIA different from a Participatory Poverty Assessment?
A Participatory Poverty Assessment characterises the nature, causes, and experience of poverty to inform policy. PIA is narrower and evaluative: it measures the impact of a specific project against defined impact questions, using community indicators and structured scoring to estimate the size and direction of change. Both use PRA tools, but PIA is organised around attributing change to an intervention.
Related methods
Sources
Catley, A., Burns, J., Abebe, D., & Suji, O. (2014). Participatory Impact Assessment: A Design Guide. Somerville, MA: Feinstein International Center, Tufts University.
Open publication referenceCopied.