Skip to content
Back to search results

Randomized Evaluation in Development

Introduced by
Esther Duflo, Abhijit Banerjee, Michael Kremer; J-PAL / IPA
Year
2003
Type
Experimental impact evaluation design
Family
Experimental causal inference
Font size

What does it do?

Randomized evaluation applies the logic of the controlled experiment to development policy: an intervention — a school grant, a deworming pill, an insurance product — is assigned at random to some units and withheld from others, so that any subsequent difference in outcomes can be attributed causally to the intervention rather than to confounding.

Championed from the early 2000s by the Abdul Latif Jameel Poverty Action Lab (J-PAL) and Innovations for Poverty Action (IPA), the approach earned its leading proponents — Esther Duflo, Abhijit Banerjee, and Michael Kremer — the 2019 Nobel Memorial Prize in Economics for transforming how anti-poverty programmes are tested.

The intuition behind the method

If you want to know whether a programme works, the honest comparison is between the people who got it and the people who would have got it but, by the flip of a coin, did not. Randomisation makes the treatment and comparison groups statistically identical on everything — observed and unobserved — except the programme itself, so the messy problem of selection (better-off or more-motivated households opting in) dissolves. The estimate is then almost embarrassingly simple: take the average outcome among those offered the programme, subtract the average outcome among those not offered it, and that difference is the causal effect.

History

Field experiments entered development economics in earnest around 2000, building on earlier social experiments in the United States (such as the negative income tax and RAND health trials) and on Michael Kremer's randomised studies of education in Kenya. The founding of J-PAL at MIT in 2003 by Esther Duflo, Abhijit Banerjee, and Sendhil Mullainathan, alongside IPA, institutionalised the method and produced hundreds of trials. Duflo, Glennerster, and Kremer's 2007 Handbook toolkit codified the practical methodology. The 2019 Nobel Memorial Prize to Banerjee, Duflo, and Kremer marked the approach's arrival at the disciplinary mainstream, even as a vigorous debate over external validity continued.

When to use it

Use a randomized evaluation when the central question is whether a specific, deliverable intervention causes a change in measurable outcomes, when ethical and operational conditions permit withholding or phasing in the programme, and when the population is large enough to detect a policy-relevant effect with adequate statistical power. It is the design of choice for testing programme prototypes and mechanisms. It is poorly suited to evaluating economy-wide policies, unique large infrastructure, or interventions whose effects unfold over decades, and it cannot by itself explain why a programme worked — questions better addressed by theory-based or mixed-methods approaches.

Limitations and considerations

  • External validity is not guaranteed: an effect measured in one context, scale, or population may not transfer to another (the site-selection and scale-up problem).
  • Many of the largest policy questions — macro policy, institutions, exchange rates — cannot be randomised at all.
  • Spillovers, general-equilibrium effects, and Hawthorne/John Henry behavioural responses can bias estimates if the design ignores them.
  • Trials are expensive and slow, and ethical or political constraints may forbid withholding treatment from a control group.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Random assignment eliminates selection bias by construction, giving the cleanest available identification of a causal effect with minimal modelling assumptions.
  • Results are transparent and credible to non-specialists: the headline estimate is a simple difference in group averages.
  • Pre-analysis plans and registration discipline the analysis, reducing the risk of specification searching and publication bias.
  • The method has built a cumulative, replicable evidence base across countries and sectors, enabling cost-effectiveness comparisons of competing programmes.

Applications

  • Testing the impact of conditional and unconditional cash transfers on schooling, consumption, and health.
  • Evaluating school inputs — textbooks, flip charts, deworming, teacher incentives — on learning and attendance.
  • Measuring the effects of microcredit access on business creation, income, and household consumption.
  • Assessing health interventions such as bed-net distribution, chlorine dispensers, and immunisation incentives.
  • Evaluating governance and behavioural-nudge interventions, from biometric monitoring to reminders for savings and loan repayment.

How it works

How it works

1

The evaluator specifies the intervention precisely, the population of interest, and the unit of randomisation — individuals, households, schools, clinics, or whole villages. The choice of unit reflects how the programme is delivered and the risk of spillovers: when treatment effects leak across nearby individuals, randomising at the cluster (e.g., village) level rather than the individual level preserves the integrity of the comparison.

2

Eligible units are allocated to treatment and control arms by an explicit random mechanism, often stratified on baseline characteristics to improve balance and precision. Random assignment is what guarantees that the two groups are comparable in expectation, so that the control group is a valid counterfactual for what would have happened to the treatment group absent the programme. Encouragement designs randomise the offer rather than take-up when forced assignment is infeasible.

3

Pre-specified outcomes are measured at baseline and endline through surveys, administrative records, or biomarkers, ideally according to a registered pre-analysis plan that fixes hypotheses and avoids data mining. Careful tracking minimises differential attrition between arms, which would reintroduce bias, and the analyst distinguishes the intention-to-treat (ITT) effect of being offered the programme from the treatment-on-the-treated (ToT or LATE) effect among those who actually took it up.

4

The average treatment effect is estimated as the difference in mean outcomes between the treatment and control groups, often via a regression that adjusts for the stratification variables and clusters standard errors at the level of randomisation. Instrumental-variables (LATE) estimators scale the ITT by the take-up rate to recover the effect on compliers, and statistical inference quantifies whether the measured difference is distinguishable from chance.

Formula
\hat{\tau} = \bar{Y}_{T} - \bar{Y}_{C}

Common pitfalls

  • Differential attrition between treatment and control arms quietly reintroduces the selection bias that randomisation was meant to remove.
  • Failing to cluster standard errors at the level of randomisation overstates precision and produces spurious significance.
  • Underpowered designs — too few clusters or units — yield noisy estimates and a high chance of missing real effects.
  • Reporting treatment-on-the-treated effects without acknowledging non-compliance, or fishing across many outcomes without correction, overstates programme impact.

Frequently asked questions

What is the difference between intention-to-treat and treatment-on-the-treated effects?

The intention-to-treat (ITT) effect compares everyone assigned to treatment against everyone assigned to control, regardless of whether they actually took up the programme; it reflects the effect of offering the programme under realistic non-compliance. The treatment-on-the-treated (ToT), recovered as a local average treatment effect (LATE) via instrumental variables, divides the ITT by the take-up rate to estimate the effect on those who actually complied. ITT is policy-relevant for roll-out decisions; ToT speaks to the effect of the treatment itself on participants.

Why randomise at the cluster level instead of the individual level?

Cluster randomisation — assigning whole villages, schools, or clinics — is used when treatment effects spill over between nearby individuals (e.g., deworming reduces transmission to untreated neighbours) or when the intervention is naturally delivered at a group level. It protects the comparison from contamination, but it reduces statistical power because outcomes within a cluster are correlated, so standard errors must be clustered and sample-size calculations must account for the intra-cluster correlation.

Does a successful trial mean the programme will work everywhere?

Not necessarily. A randomized evaluation has strong internal validity — it credibly estimates the effect in the studied context — but external validity, the transfer of that effect to other populations, scales, or settings, must be argued separately. Replication across sites, attention to the mechanism behind the effect, and theory about which contextual features matter are needed before generalising. Effects can also change at scale through general-equilibrium and implementation effects that a small trial cannot capture.

Related methods

Sources

  1. Banerjee, A. V., & Duflo, E. (2009). The Experimental Approach to Development Economics. Annual Review of Economics, 1, 151–178.

    Open DOI reference
    Copied.
  2. Duflo, E., Glennerster, R., & Kremer, M. (2007). Using Randomization in Development Economics Research: A Toolkit. Handbook of Development Economics, 4, 3895–3962.

    Open DOI reference
    Copied.