Skip to content

Compare methods

Compare source explanations, strengths, limitations and references for up to four methods.

9 results
Criterion EM Algorithm MICE Maximum Likelihood Estimation
Overview
  • The Expectation-Maximization (EM) algorithm is an iterative optimization procedure for finding maximum likelihood or maximum a posteriori estimates of parameters in statistical models with latent variables or missing data. Introduced by Dempster, Laird, and Rubin in their landmark 1977 paper, EM alternates between computing the expected complete-data log-likelihood (E-step) and maximizing it with respect to the parameters (M-step), guaranteeing monotone non-decreasing likelihood at each iteration.
  • Multivariate Imputation by Chained Equations (MICE) is an iterative procedure for handling missing data in multivariate datasets. Introduced by Stef van Buuren and Karin Groothuis-Oudshoorn through the R package mice (2011), the algorithm fills each missing variable using a separate regression model conditioned on all other variables, cycling through variables repeatedly until the imputed values converge. The result is m completed datasets that are analysed separately and combined using Rubin's rules.
  • Maximum Likelihood Estimation (MLE) is a general-purpose parametric method for estimating the unknown parameters of a statistical model by finding the parameter values that make the observed data most probable. Formalized by R. A. Fisher in his landmark 1922 paper in the Philosophical Transactions of the Royal Society, MLE has become the dominant parameter-estimation paradigm in modern statistics and is the foundational engine behind logistic regression, generalized linear models, structural equation modeling, and virtually all parametric inference procedures.
Use and interpretation
  • Use the EM algorithm when data contain missing values, hidden states, or latent variables and you need maximum likelihood estimates. It is well-suited to Gaussian mixture models, hidden Markov models, factor analysis, and missing-data imputation. Key assumptions include that missing data are missing at random (MAR) and that the complete-data model is tractable. Limitations include sensitivity to initialization, possible convergence to local maxima, and slow convergence near saddle points. Alternatives include gradient-based optimization with numerical integration, variational inference, or MCMC methods.
  • MICE is appropriate when data are missing at random (MAR) and missingness occurs across multiple variables simultaneously. It suits continuous, binary, ordinal, and count outcomes through its flexible per-variable model specification. It is less suitable for data missing not at random (MNAR) without auxiliary modelling, for very high-dimensional datasets where the imputation model becomes unwieldy, or when missingness rates exceed 50% in key variables. Single imputation or complete-case analysis may suffice under MCAR with low missingness rates.
  • MLE is applicable whenever you can write down an explicit parametric density or mass function for the data-generating process and the observations are independent (or have a tractable dependence structure). It is the standard estimation method for linear and generalised linear models, survival models, mixed-effects models, factor analysis, SEM, and discrete-choice econometric models. Core assumptions include: (1) the model family is correctly specified; (2) observations are independent; (3) the parameter is identifiable; (4) the true parameter lies in the interior of the parameter space. With small samples the MLE may be biased; alternatives such as restricted MLE (REML) or bias-corrected MLE address this. If a prior distribution over parameters is available, Bayesian maximum a posteriori (MAP) estimation extends MLE naturally.
Parametric
  • Yes
Output
  • θ (parameter vector)
Strengths
  • Guaranteed monotone increase in observed-data likelihood at every iteration, ensuring stable convergence
  • Converts a hard missing-data problem into a sequence of tractable complete-data optimizations
  • Broadly applicable across exponential-family models, mixture models, and latent-variable frameworks
  • Closed-form M-step updates are available for many standard models, making implementation efficient
  • Handles mixed variable types (continuous, binary, ordinal, count) via tailored per-column models
  • Properly propagates uncertainty through multiple imputed datasets and Rubin's pooling rules
  • Flexible predictor specification allows domain knowledge and auxiliary variables to improve imputation quality
  • Well-validated R implementation (mice package) with extensive diagnostics and active maintenance
  • Asymptotically optimal: achieves the Cramér–Rao lower bound, making it the most efficient consistent estimator in large samples.
  • Invariance property: if θ̂ is the MLE of θ, then g(θ̂) is the MLE of g(θ) for any function g — making transformations of estimates straightforward.
  • Universal framework: a single principle drives estimation in linear models, GLMs, survival analysis, SEM, time-series models, and more.
  • Automatic standard errors: the observed or expected Fisher information matrix directly yields asymptotic variance estimates, confidence intervals, and Wald test statistics.
  • Software support: implemented in every major statistical package (R, Python/statsmodels, SAS, Stata, SPSS) often as the default estimation engine.
Limitations
  • Convergence to a local maximum rather than the global maximum when the likelihood surface is multimodal
  • Can exhibit very slow (linear) convergence when the fraction of missing information is large
  • Requires the analyst to specify the complete-data model and the distribution of latent variables
  • Standard EM does not directly provide the observed-data covariance matrix of the estimates; a supplementary step (e.g., Louis's method or the SEM algorithm) is needed for standard errors
  • Assumes data are missing at random (MAR); violations require sensitivity analyses or pattern-mixture extensions
  • Computational cost grows with the number of variables, iterations, and imputations (m)
  • Convergence of the Gibbs-like sampler is not mathematically guaranteed for incompatible conditional models
  • Model specification (predictor selection, imputation method per variable) requires careful analyst judgment
  • Requires correct model specification: if the assumed density family is wrong, MLEs are inconsistent and inferences are invalid.
  • May be biased in small samples: the MLE is asymptotically unbiased but can carry non-negligible finite-sample bias (e.g., the MLE of σ² in a normal model is biased downward).
  • Numerically sensitive in complex models: high-dimensional or multimodal likelihood surfaces can cause convergence failures or convergence to local maxima.
  • Does not incorporate prior information: unlike Bayesian estimation, MLE treats parameters as fixed unknowns and provides no mechanism to include substantive prior knowledge.
Common pitfalls
  • Using a single random initialization risks terminating at a poor local maximum; always run multiple restarts and select the solution with the highest final log-likelihood
  • Declaring convergence too early by monitoring only parameter change rather than the observed-data log-likelihood can yield inaccurate estimates
  • Applying EM when data are Missing Not At Random (MNAR) without model adjustment can introduce systematic bias in estimates
  • Ignoring the label-switching problem in mixture models leads to uninterpretable component assignments even after correct convergence
  • Using too few imputed datasets (m < 5) underestimates standard errors and inflates false-positive rates
  • Including the outcome variable in the imputation model is essential; omitting it biases regression coefficients toward null
  • Ignoring interaction terms or non-linear relationships in the imputation model can distort downstream analyses
  • Failing to check convergence via trace plots may lead to pooling results from non-converged chains
  • Ignoring model misspecification — always check goodness-of-fit diagnostics; a well-converged MLE from a wrong model is still wrong.
  • Treating numerical convergence as verification of a global maximum — inspect the likelihood surface or try multiple starting values in complex models.
  • Using MLE standard errors in small or sparse samples without correction (e.g., Firth's penalised likelihood for logistic regression with rare events).
  • Conflating the likelihood with a probability: L(θ | x) is not a probability distribution over θ and integrates to neither 0 nor 1 in general.
Introduced by
  • Dempster, Laird & Rubin
  • Stef van Buuren & Karin Groothuis-Oudshoorn
  • R. A. Fisher
Year
  • 1977
  • 2011
  • 1922