Skip to content

Compare methods

Compare source explanations, strengths, limitations and references for up to four methods.

8 results
Criterion Network Meta-Analysis Meta-Regression
Overview
  • Network meta-analysis (NMA) is a systematic method for comparing multiple interventions simultaneously within a single analytical framework, incorporating both direct evidence (head-to-head trials) and indirect evidence (comparisons via common comparators). First formalized by Lumley in 2002, NMA allows researchers to rank treatments and quantify comparative effectiveness even when some treatment pairs have never been directly studied.
  • Meta-regression is a statistical technique that extends conventional meta-analysis by regressing study-level effect sizes on one or more study characteristics (moderators) to explain between-study heterogeneity. Formalized by Thompson and Higgins in 2002, it uses weighted least squares — weighting each study by the inverse of its variance — within a mixed-effects framework, allowing researchers to identify which study features systematically account for variation in observed effects across the literature.
Use and interpretation
  • Use NMA when (1) multiple interventions for the same condition exist, (2) researchers and clinicians need to rank or choose between them, (3) direct head-to-head trials are sparse or absent for some comparisons, (4) a connected network exists (treatments linked directly or indirectly), and (5) you can assess and justify the assumption of consistency. Common applications include drug comparisons (antidepressants, antihypertensives), surgical techniques, rehabilitation interventions, and preventive strategies. NMA is particularly valuable in health technology assessment (HTA) to inform reimbursement decisions across multiple options.
  • Meta-regression is appropriate when a meta-analysis reveals significant heterogeneity (I-squared > 25–50%) and the analyst has a priori hypotheses about study-level moderators that may explain it. It requires a sufficient number of studies — commonly at least ten studies per moderator to avoid overfitting. Assumptions include linearity of moderator effects and availability of moderator data for most included studies. When moderators are individual-level rather than study-level, individual participant data (IPD) meta-analysis is preferable. Subgroup analysis is a simpler alternative when moderators are categorical and studies are few.
Output
  • Regression coefficients explaining between-study variance
Strengths
  • Enables ranking of all treatments on a single efficacy scale, even when direct trial evidence is absent for some pairs.
  • Increases statistical power by borrowing strength from indirect comparisons, improving precision of estimates.
  • Provides a comprehensive evidence synthesis for guideline and policy development, reducing reliance on arbitrary or incomplete pairwise meta-analyses.
  • Identifies network inconsistencies that may signal heterogeneity, study design bias, or unmeasured effect modifiers.
  • Integrates multi-arm trials naturally, avoiding the need to break trials arbitrarily into separate pairwise comparisons.
  • Quantifies how much of between-study heterogeneity is explained by specific study characteristics
  • Accommodates multiple moderators simultaneously in a single model
  • Weights studies by precision, making full use of available statistical information
  • Provides an R-squared analog that communicates explanatory power in an interpretable metric
Limitations
  • Requires strong assumption of consistency (transitivity): that treatment effects are comparable across different trial populations and contexts. Violation invalidates rankings.
  • Results depend on the evidence base; sparse networks or unconnected subgraphs reduce precision and may lead to unreliable indirect estimates.
  • Computational complexity increases with network size, and Bayesian NMA requires careful specification of prior distributions, which can influence results.
  • Assumes all available trials are identified; missing studies (especially negative ones) can bias results via small-study effects.
  • Requires a large number of studies (typically 10+ per moderator) to yield reliable estimates
  • Ecological fallacy risk: study-level associations may not reflect individual-level relationships
  • Results are sensitive to the method chosen for estimating residual between-study variance
  • Cannot establish causality; confounding among study characteristics is common
Common pitfalls
  • Interpreting SUCRA as absolute probability of superiority rather than relative rank within the study network. SUCRA 75% does not mean 75% chance of being best; it reflects ranking position, and uncertainty may be high.
  • Assuming consistency without testing. Always check consistency explicitly via consistency plots or models that allow inconsistency parameters.
  • Applying NMA to networks with very sparse comparisons (e.g., only one trial per edge). Indirect estimates become unreliable; report with appropriate caution.
  • Ignoring subgroup or meta-regression evidence that suggests treatment effects differ across populations. If consistency is violated, stratified NMA may be needed.
  • Data dredging: testing many moderators without a priori hypotheses inflates Type I error — pre-registration of moderators is strongly recommended
  • Treating aggregate study characteristics as proxies for individual-level effects (aggregation bias) can produce misleading conclusions
  • Including studies with missing moderator data without appropriate imputation introduces selection bias
  • Interpreting a significant moderator effect as causal without accounting for correlation among study-level covariates
Introduced by
  • Lumley (2002)
  • Simon Thompson & Julian Higgins
Year
  • 2002
  • 2002