Synthetic Control for Health Policy
- Introduced by
- Alberto Abadie & Javier Gardeazabal; Alberto Abadie, Alexis Diamond & Jens Hainmueller
- Year
- 2010
- Type
- Comparative case-study design constructing a weighted comparator for an aggregate health-policy unit
- Family
- Causal inference / comparative case studies for policy evaluation
What does it do?
The synthetic control method evaluates the effect of a population-health policy implemented in a single aggregate unit — a state, country, or region — by building a data-driven comparator from a pool of untreated units.
When a policy such as a tobacco tax, an alcohol-pricing law, a smoking ban, or a health-insurance expansion is enacted in one place, no single other place is a perfect counterfactual. The method instead forms a synthetic version of the treated unit as a weighted average of donor units chosen so that the synthetic closely tracks the treated unit's outcome and predictors before the policy. The post-intervention gap between the real unit and its synthetic twin estimates the policy's effect. Introduced by Abadie and Gardeazabal and formalized by Abadie, Diamond and Hainmueller — whose canonical application is California's Proposition 99 tobacco-control program — it has become a leading design for evaluating health policies at the population level, with placebo tests providing inference.
The intuition behind the method
Imagine California passes a sweeping tobacco-control program and you want to know how many cigarette packs it prevented. You cannot rerun history without the program, and no other single state matches California well enough to serve as the comparison — California is too big, too unusual, too distinctive. The synthetic control trick is to build a fake California out of the other states: take a blend of, say, some Utah, some Nevada, some Colorado, weighted so that this combination smoked almost exactly like California did in the years before the program. If the recipe reproduces California's pre-program cigarette sales and the things that drive them, it is a credible stand-in for what California would have done without the program. After the program starts, you watch the gap open up between real California and synthetic California: that widening gap is your estimate of the policy's effect. To judge whether the gap is real or just noise, you rerun the whole exercise pretending each other state was the one that got the policy, and check whether the genuine California gap is unusually large compared to these placebos.
History
The synthetic control method was introduced by Alberto Abadie and Javier Gardeazabal in their 2003 American Economic Review study of the economic costs of conflict in the Basque Country, which built a synthetic comparator region from other Spanish regions. Abadie, Diamond and Hainmueller's 2010 Journal of the American Statistical Association paper formalized the estimator, established the convex-weighting framework, and introduced placebo-based inference, using California's Proposition 99 tobacco-control program as the leading application — a population-health policy evaluation that has become the method's textbook case. The approach spread rapidly through health-policy research, where it has been used to evaluate tobacco and alcohol controls, sugary-drink taxes, gun and drug policies, and insurance reforms. Subsequent methodological work has added generalizations such as the synthetic difference-in-differences, augmented synthetic control, and methods for staggered multi-unit adoption, but the original framework remains the standard for single-unit population-health policy evaluation.
When to use it
Use synthetic control for health policy when a single aggregate unit adopts a discrete population-health intervention and you have panel data on the outcome and its predictors for that unit and a pool of comparable untreated units, with a substantial pre-intervention period. It shines for state- or country-level policies such as tobacco taxes and smoking bans, minimum alcohol pricing, soda taxes, and insurance expansions, where there are few treated units and no obvious single control. It is preferable to a simple before-after or difference-in-differences comparison when no single donor is a good match and the parallel-trends assumption is doubtful, because it transparently constructs the best available comparator and shows its pre-period fit. It is ill-suited when the pre-intervention period is short, when no convex combination of donors can reproduce the treated unit, when donors are contaminated by the same or spillover policies, or when many units are treated at once (where panel or staggered-adoption estimators are better).
Limitations and considerations
- Requires a long, stable pre-intervention period; short pre-periods make the fit and the counterfactual unreliable.
- Fails when no weighted combination of available donors can reproduce the treated unit's pre-policy trajectory.
- Sensitive to the composition of the donor pool and to contamination of donors by spillovers or concurrent policies.
- Inference rests on placebo permutation rather than conventional sampling theory, and is weak when the donor pool is small.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Constructs a transparent, data-driven comparator for a single treated unit, with pre-period fit visible as a check on credibility.
- Restricts to convex combinations of donors, avoiding the extrapolation that can plague regression-based counterfactuals.
- Provides permutation-based placebo inference suited to the one-treated-unit setting where standard errors do not apply.
- Yields an intuitive, communicable result — the gap between the real unit and its synthetic twin — well suited to policy audiences.
Applications
- Estimating the effect of a state or national tobacco-control program on per-capita cigarette consumption and smoking-related mortality.
- Evaluating minimum unit pricing or other alcohol policies on alcohol consumption and alcohol-attributable harm.
- Assessing sugary-drink or soda taxes on purchases and dietary outcomes at the city or state level.
- Measuring the impact of a health-insurance expansion or reform on coverage and access in the adopting state.
- Studying the population-health consequences of regional drug, firearm, or public-health regulations relative to a constructed comparator.
How it works
How it works
The analyst first identifies the treated unit that adopted the health policy and assembles a donor pool of comparable units that did not, all observed over a panel that spans well before and after the intervention. For each unit they collect the outcome of interest — per-capita cigarette sales, alcohol-attributable mortality, uninsured rates, and so on — along with predictors thought to drive that outcome. Donors must be plausibly unaffected by the policy and not subject to large idiosyncratic shocks of their own; Abadie, Diamond and Hainmueller stress that a long, clean pre-intervention period and a sensible donor pool are prerequisites for a credible synthetic control.
Formula
\text{treated } j=1;\ \text{donors } j=2,\dots,J{+}1;\ \ Y_{jt},\ \mathbf{X}_j\ (t=1,\dots,T_0,\dots,T)The heart of the method is choosing non-negative weights on the donor units that sum to one, so that the weighted combination best reproduces the treated unit's pre-intervention predictors and outcome trajectory. These weights are found by minimizing the discrepancy between the treated unit's pre-period characteristics and those of the weighted donors, with a second layer of weights determining how much each predictor matters. Constraining the weights to be non-negative and to sum to one — a convex combination — guards against extrapolation, so the synthetic unit stays within the range of the observed donors, a key safeguard Abadie and colleagues emphasize over conventional regression.
Formula
W^{*}=\arg\min_{W}\ (\mathbf{X}_1-\mathbf{X}_0 W)^{\top}V(\mathbf{X}_1-\mathbf{X}_0 W),\quad w_j\ge 0,\ \textstyle\sum_j w_j=1Applying the estimated weights to the donor units' outcomes yields the synthetic control: the trajectory the treated unit's outcome would have followed, according to the data, had the policy not been enacted. In Abadie and Gardeazabal's original Basque application this synthetic region reproduced the treated region's economic path before the conflict; in the health-policy setting it reproduces, for example, the pre-policy path of smoking or drinking. A good synthetic unit hugs the treated unit closely throughout the pre-intervention window, which is the visual and quantitative evidence that the comparator is credible.
Formula
\hat{Y}_{1t}^{N}=\sum_{j=2}^{J+1} w_j^{*}\, Y_{jt}The estimated policy effect is the gap between the treated unit's actual outcome and its synthetic counterpart in each post-intervention period. If the two tracked each other closely before the policy and then diverge afterward, the divergence is attributed to the policy. Abadie, Diamond and Hainmueller report this gap as the central result — for California's Proposition 99, a growing shortfall in per-capita cigarette sales relative to synthetic California — and recommend examining its size and persistence over the post-period rather than a single summary number.
Formula
\hat{\tau}_{1t}=Y_{1t}-\sum_{j=2}^{J+1} w_j^{*}\, Y_{jt},\quad t>T_0Because there is only one treated unit, classical standard errors do not apply, so inference is done by placebo tests. The analyst reassigns the policy in turn to each donor, constructs a synthetic control for that placebo unit, and computes its gap. Comparing the treated unit's post-period gap — typically scaled by pre-period fit using the ratio of post- to pre-intervention root-mean-squared prediction error — against the distribution of placebo gaps gives a permutation-style p-value. If the genuine effect is large relative to placebos that fit equally well, the result is judged unlikely to be chance, the inferential logic introduced in the tobacco-control paper.
Formula
p=\dfrac{1}{J+1}\sum_{j} \mathbb{1}\!\left[\dfrac{\mathrm{RMSPE}_j^{post}}{\mathrm{RMSPE}_j^{pre}}\ge \dfrac{\mathrm{RMSPE}_1^{post}}{\mathrm{RMSPE}_1^{pre}}\right]Common pitfalls
- Accepting results despite poor pre-intervention fit, where the synthetic unit fails to track the treated unit before the policy.
- Including donors that were themselves affected by the policy or by similar concurrent interventions, biasing the comparator.
- Cherry-picking predictors or donor pools until the desired post-period gap appears, undermining the placebo logic.
- Reporting the gap as significant without placebo tests, or ignoring that a tiny donor pool yields uninformative permutation p-values.
Frequently asked questions
How is synthetic control different from difference-in-differences?
Difference-in-differences compares the treated unit to one or more controls under the assumption that their outcomes would have moved in parallel absent the policy, weighting controls equally or by a fixed rule. Synthetic control instead estimates data-driven weights so that a convex combination of donors reproduces the treated unit's pre-intervention path, then reads the effect as the post-period gap. This makes the comparator's quality visible through pre-period fit and avoids assuming parallel trends with an arbitrary control group. Abadie, Diamond and Hainmueller present it precisely as a more credible alternative when no single control matches the treated unit.
Why does it use placebo tests instead of standard errors?
With only one treated unit there is no sampling distribution of treated outcomes to support conventional standard errors. Instead, inference is done by reassigning the policy to each donor, building a synthetic control for that placebo, and computing its gap. The treated unit's effect is judged significant if it is extreme relative to the distribution of placebo gaps, typically after scaling by pre-period fit. This permutation logic, introduced in the California tobacco study, asks how often a gap as large as the observed one arises by chance among units that were not actually treated.
What makes a synthetic control credible for a health policy?
Three things. First, a close pre-intervention fit: the synthetic unit should track the treated unit's outcome and predictors for many periods before the policy. Second, a clean donor pool of units that did not adopt the policy or a similar one and were not affected by spillovers. Third, robustness: the result should survive reasonable changes to the predictor set and donor pool and should pass placebo tests. When the pre-period fit is poor or no donor combination can reproduce the treated unit, Abadie and colleagues advise against drawing causal conclusions from the method.
Related methods
Sources
Abadie, A., & Gardeazabal, J. (2003). The Economic Costs of Conflict: A Case Study of the Basque Country. American Economic Review, 93(1), 113-132.
Open DOI referenceCopied.Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program. Journal of the American Statistical Association, 105(490), 493-505.
Open DOI referenceCopied.