Skip to content

Compare methods

Compare source explanations, strengths, limitations and references for up to four methods.

8 results
Criterion Multiple Imputation Propensity Score Matching
Overview
  • Multiple Imputation (MI), formally introduced by Donald B. Rubin in 1987, is a principled statistical procedure for handling missing data. Rather than replacing each missing value once, MI fills the gaps m times — each time drawing plausible values from the posterior predictive distribution of the missing data — producing m complete datasets. Each dataset is analysed independently, and the results are combined into a single set of estimates using Rubin's pooling rules. The MICE variant (Multivariate Imputation by Chained Equations), popularised by van Buuren and Groothuis-Oudshoorn (2011), extends the approach to mixed variable types by imputing each variable in turn through a sequence of conditional regression models.
  • Propensity score matching (PSM) is a method for reducing confounding bias in observational studies by balancing baseline characteristics between treatment groups, simulating randomization. Developed by Rosenbaum and Rubin (1983), it estimates the probability of receiving treatment given observed covariates, then matches or weights treated and control individuals with similar treatment probabilities. Widely used in medicine, epidemiology, and policy evaluation when randomized trials are infeasible or unethical, enabling estimation of treatment effects while controlling for selection bias.
Use and interpretation
  • Multiple imputation is appropriate when a dataset has missing values and the missingness mechanism is plausibly MAR or MCAR. It applies to continuous, categorical, binary, ordinal, and mixed-type variables and fits cross-sectional, longitudinal, and panel data structures. A minimum sample size of approximately 30 is required. The method suits descriptive, predictive, and relational research purposes. When missingness is MNAR, standard multiple imputation is insufficient and must be supplemented by a sensitivity analysis or a selection model. Listwise deletion (complete-case analysis) is acceptable only when data are MCAR and the proportion of missing data is very small; otherwise MI is the preferred approach.
  • Use propensity score matching when conducting observational studies and unable to randomize, but all confounders are measured. Medical research: evaluating treatment efficacy (surgery vs conservative care) comparing outcomes for matched treated/untreated patients with similar baseline risk. Policy evaluation: assessing program impact (e.g., vocational training on employment) comparing participants and matched nonparticipants. Epidemiology: estimating disease risk associated with exposure while controlling baseline characteristics. Prerequisite: must have collected sufficient baseline data on confounders before treatment assignment; unmeasured confounding (e.g., unmeasured disease severity) remains a threat.
Sample size
  • 30
Strengths
  • Produces unbiased estimates under MAR, unlike listwise deletion which requires the stronger MCAR assumption.
  • Propagates uncertainty from missing data into standard errors and confidence intervals through Rubin's pooling rules, avoiding the false precision of single imputation.
  • Handles mixed variable types — continuous, binary, ordinal, nominal — within the same MICE framework.
  • Compatible with virtually any downstream analysis model, requiring no modification to the intended analysis.
  • Well-supported in standard statistical software (R mice, SPSS MULTIPLE IMPUTATION, Python IterativeImputer / miceforest).
  • Reduces confounding bias: by creating groups balanced on measured confounders, PSM mimics randomization for those variables, enabling causal inference.
  • Interpretable: propensity score is a single scalar collapsing many covariates; transparent how covariate balance achieved.
  • Flexible: works with binary or multiple treatments; extends to continuous treatments (generalized propensity score).
  • Diagnostic: covariate balance assessment (SMD, love plots) provides clear evidence of bias reduction; imbalance visible before outcome analysis.
  • Doubly robust: when combined with outcome regression, the estimator is consistent if either the propensity model or outcome model is correct (resilience to misspecification).
Limitations
  • Requires the MAR or MCAR assumption; cannot correct for MNAR missingness without additional modelling.
  • Computationally more demanding than single imputation or listwise deletion, especially with large m or large datasets.
  • The quality of imputed values depends on the imputation model — omitting strong predictors of missingness from the imputation model biases results.
  • With high fractions of missing data (above ~50 %), even correctly specified multiple imputation may yield wide confidence intervals.
  • Conditional independence assumption: valid only if all relevant confounders are measured. Unmeasured confounding (e.g., unmeasured disease severity, patient preference) remains a threat; PSM cannot detect or correct it.
  • Inefficiency: matching reduces sample size (unmatched individuals discarded); matched analyses are less statistically powerful than unmatched regression.
  • Limited overlap: if treated and untreated populations have very different covariate distributions, overlap region is small; estimates restricted to overlap region only.
  • Model dependence: propensity score estimates depend on correct specification of logistic regression; misspecification (omitted interactions, nonlinear relationships) yields poor balance.
  • Loss of information: one-to-one matching discards potentially informative unmatched individuals; many-to-one matching or weighting retain all data.
Common pitfalls
  • Setting m too low — a common rule of thumb is m ≥ 5, but with non-trivial missing fractions a much higher m (e.g., m ≥ 100 × fraction missing) is needed to achieve stable, reproducible estimates.
  • Omitting the outcome variable or other important predictors from the imputation model produces biased imputations; the imputation model must be at least as rich as the analysis model.
  • Applying multiple imputation when the mechanism is MNAR without a supplementary sensitivity analysis gives results that appear valid but may be substantially biased.
  • Pooling results with Rubin's rules assumes that the analysis model is the same for all m datasets and that convergence of the MICE algorithm has been checked — inspect trace plots of imputed means and variances to verify convergence.
  • Matching on the outcome: biases estimates toward zero (reduces power). Propensity score modeling should include confounders (variables predicting both treatment and outcome), not mediators or consequences.
  • Failing to check overlap: proceeding with matching when treated/control propensity distributions don't overlap. Always check and report common support; estimates outside overlap are unreliable extrapolations.
  • Ignoring covariate balance: assuming matching worked without examining standardized mean differences. Always report balance diagnostics (SMD plots, love plots); imbalance indicates need for alternative approach.
  • Analyzing full sample instead of matched sample: if you match then analyze all original data, you ignore the matching (defeats purpose). Analyze matched sample only, or use weighting on full sample.
  • P-hacking with multiple propensity models: fitting different propensity models and selecting the one yielding best balance or strongest treatment effect. Specify propensity model a priori based on theory; use cross-validation.
  • Causal claims without temporal precedence: propensity score reduces confounding by measured covariates, but causality requires treatment before outcome. Reverse causality still threatens validity.
Introduced by
  • Donald B. Rubin
  • Paul Rosenbaum and Donald Rubin
Year
  • 1987
  • 1983