Skip to content

Compare methods

Compare source explanations, strengths, limitations and references for up to four methods.

7 results
Criterion Individual Patient Data Meta-Analysis Network Meta-Analysis
Overview
  • Individual patient data meta-analysis (IPD-MA) is a systematic synthesis method where researchers obtain and analyze raw data at the patient level from multiple randomized controlled trials, rather than relying on published summary statistics (aggregate data). Pioneered by the Cochrane Collaboration and formalized by Stewart, Clarke, and Riley, IPD-MA is considered the gold standard for evidence synthesis because it enables consistent outcome definition across trials, robust subgroup analysis, and detection of treatment-covariate interactions. Though time-intensive and resource-demanding, IPD-MA provides the most reliable estimates of intervention effects and is preferred for critical clinical decisions, particularly for identifying which patients benefit most from treatment.
  • Network meta-analysis (NMA) is a systematic method for comparing multiple interventions simultaneously within a single analytical framework, incorporating both direct evidence (head-to-head trials) and indirect evidence (comparisons via common comparators). First formalized by Lumley in 2002, NMA allows researchers to rank treatments and quantify comparative effectiveness even when some treatment pairs have never been directly studied.
Use and interpretation
  • Conduct IPD-MA when (1) a precise, subgroup-specific understanding of treatment effects is critical for clinical decision-making (e.g., identifying which cancer patients benefit from a new drug), (2) published summary-level meta-analysis shows heterogeneity or conflicting results and IPD analysis might clarify mechanisms or identify moderators, (3) important subgroup effects are suspected but not consistently reported across trials, (4) you can secure reasonable data response from trial authors (typically 50-80% of trials), or (5) you have resources (time, funding, data management expertise) to obtain, harmonize, and analyze patient-level data. Common applications include cancer trials, cardiovascular interventions, and chronic disease management, where precision medicine questions are important. Avoid IPD-MA if data access is severely restricted or if no meaningful subgroup questions exist.
  • Use NMA when (1) multiple interventions for the same condition exist, (2) researchers and clinicians need to rank or choose between them, (3) direct head-to-head trials are sparse or absent for some comparisons, (4) a connected network exists (treatments linked directly or indirectly), and (5) you can assess and justify the assumption of consistency. Common applications include drug comparisons (antidepressants, antihypertensives), surgical techniques, rehabilitation interventions, and preventive strategies. NMA is particularly valuable in health technology assessment (HTA) to inform reimbursement decisions across multiple options.
Strengths
  • Enables standardized outcome definition and consistent subgroup analysis across heterogeneous trials, increasing transparency and reducing bias from selective reporting.
  • Allows investigation of treatment-by-covariate interactions, identifying which patient subgroups benefit most—essential for precision medicine and personalized treatment recommendations.
  • Typically yields more precise effect estimates than aggregate meta-analysis by using all available data and correcting for trial-level clustering.
  • Detects differential treatment effects that summary-level meta-analysis may miss, particularly when subgroup effects are small or inconsistently reported.
  • Enables investigation of trial-level heterogeneity: trial design features, implementation fidelity, or patient population characteristics that explain outcome variation.
  • Enables ranking of all treatments on a single efficacy scale, even when direct trial evidence is absent for some pairs.
  • Increases statistical power by borrowing strength from indirect comparisons, improving precision of estimates.
  • Provides a comprehensive evidence synthesis for guideline and policy development, reducing reliance on arbitrary or incomplete pairwise meta-analyses.
  • Identifies network inconsistencies that may signal heterogeneity, study design bias, or unmeasured effect modifiers.
  • Integrates multi-arm trials naturally, avoiding the need to break trials arbitrarily into separate pairwise comparisons.
Limitations
  • Substantially more time-consuming and resource-intensive than aggregate meta-analysis. Obtaining data from trial authors, curating, and harmonizing patient datasets requires 12-24 months.
  • Dependent on data availability: trials may refuse to share data due to privacy concerns, lack of resources, or loss of data. Non-response can bias results if negative trials are less likely to share.
  • Data quality may vary across trials; incomplete or inconsistently measured variables limit analyses. Missing data patterns differ by trial and require careful handling.
  • Statistical complexity increases; fitting hierarchical regression models with multiple interactions requires statistical expertise and can be prone to overfitting if not carefully planned.
  • Requires strong assumption of consistency (transitivity): that treatment effects are comparable across different trial populations and contexts. Violation invalidates rankings.
  • Results depend on the evidence base; sparse networks or unconnected subgraphs reduce precision and may lead to unreliable indirect estimates.
  • Computational complexity increases with network size, and Bayesian NMA requires careful specification of prior distributions, which can influence results.
  • Assumes all available trials are identified; missing studies (especially negative ones) can bias results via small-study effects.
Common pitfalls
  • Conducting exploratory subgroup analyses without pre-specification. Many subgroup interactions tested (e.g., treatment × age, treatment × sex, treatment × baseline severity) increase false-positive rate. Pre-register analyses and adjust p-values for multiple testing.
  • Assuming missing data are missing completely at random (MCAR) without testing. If sicker patients are more likely to have missing outcome data, analysis may overestimate treatment benefit. Conduct sensitivity analyses under different missing data assumptions.
  • Over-interpretation of subgroup effects in small subgroups. If only 50 patients exist in a subgroup, credible intervals around the subgroup effect estimate will be wide; avoid strong clinical recommendations based on imprecise estimates.
  • Ignoring trial-level clustering. Analyzing all patient data as if independent (rather than using mixed models accounting for trial clustering) underestimates variance and inflates Type I error rate.
  • Interpreting SUCRA as absolute probability of superiority rather than relative rank within the study network. SUCRA 75% does not mean 75% chance of being best; it reflects ranking position, and uncertainty may be high.
  • Assuming consistency without testing. Always check consistency explicitly via consistency plots or models that allow inconsistency parameters.
  • Applying NMA to networks with very sparse comparisons (e.g., only one trial per edge). Indirect estimates become unreliable; report with appropriate caution.
  • Ignoring subgroup or meta-regression evidence that suggests treatment effects differ across populations. If consistency is violated, stratified NMA may be needed.
Introduced by
  • Cochrane Collaboration, Pioneered by Stewart & Clarke
  • Lumley (2002)
Year
  • 1990s
  • 2002