Skip to content

Compare methods

Compare source explanations, strengths, limitations and references for up to four methods.

10 results
Criterion Diagnostic Accuracy Study Design Randomized Controlled Trial Cohort Study Design
Overview
  • A diagnostic accuracy study evaluates how well a new diagnostic test (or biomarker, imaging modality, clinical assessment) detects the presence or absence of disease compared to a reference standard (gold standard). Standardized since 2003 by the STARD (Standards for Reporting of Diagnostic Accuracy Studies) initiative, diagnostic accuracy studies are fundamental to clinical medicine, determining whether and how new tests can improve patient diagnosis and treatment.
  • A randomized controlled trial (RCT) is the gold standard experimental design in clinical and health research, in which participants are randomly allocated to a treatment group or a control group so that the effect of an intervention can be measured with the highest possible degree of internal validity. The modern parallel-group RCT was formalized by Austin Bradford Hill and the Medical Research Council in their landmark streptomycin trial of 1948, and its reporting is governed today by the CONSORT 2010 guidelines (Schulz et al., 2010).
  • A cohort study follows a group of individuals forward in time from exposure to outcome. Exposed and unexposed participants (or participants with differing exposure levels) are enrolled at baseline, characterized, and observed prospectively until the outcome occurs or the study ends. Cohort studies are fundamental to epidemiology and are the design of choice for establishing causal associations when randomized trials are infeasible or unethical.
Use and interpretation
  • Use diagnostic accuracy studies when: (1) developing or evaluating a new diagnostic test, imaging modality, or biomarker, (2) comparing two diagnostic tests to determine which is superior, (3) determining whether a test can accurately detect disease in clinical populations, (4) assessing whether test performs similarly in subgroups (by age, sex, disease stage, comorbidity), (5) informing clinical practice guidelines on test adoption, (6) evaluating screening tests in asymptomatic populations, (7) assessing diagnostic tests in clinical decision-making (does it guide treatment choices?).
  • Use an RCT when you need to establish a causal relationship between an intervention and an outcome under controlled conditions. The design is appropriate when participants can be ethically randomized, when you can exercise sufficient control over treatment delivery and follow-up, and when the sample is large enough for adequate statistical power (determined by a pre-study power analysis). Minimum recommended sample is approximately 30 participants in total (n ≥ 10 per group as an absolute lower bound). The design accommodates continuous, binary, and ordinal outcome variables and both cross-sectional (single time-point) and longitudinal (repeated-measure) structures. Key assumptions are: randomization eliminates selection bias; blinding reduces measurement and performance bias; intention-to-treat and per-protocol analyses are both reported; and sample size is justified by a formal power analysis.
  • Use cohort studies when: (1) studying rare exposures (e.g., occupational hazards affecting few people), (2) establishing temporal relationships between exposure and outcome, (3) studying multiple outcomes from a single exposure (e.g., smoking → cancer, heart disease, COPD), (4) evaluating long-term effects over years or decades, (5) assessing natural history and incidence of disease, (6) randomized trials are unethical (e.g., exposing people to harmful substances), (7) outcomes are common, making case-control less efficient.
Parametric
  • No
Groups and design
  • 2+
Sample size
  • 10
Strengths
  • Direct and transparent: clearly answers 'Does this test accurately detect disease?'
  • Standardized methodology: STARD guidelines ensure consistent reporting and prevent bias.
  • Practical relevance: results directly inform clinical decision-making and test adoption.
  • Multiple metrics: sensitivity, specificity, predictive values, LRs, and ROC curves provide nuanced picture of test performance.
  • Subgroup assessment: can evaluate whether test works equally well across diverse populations (important for equity).
  • Randomization breaks confounding at the design stage, enabling the strongest possible causal inference without adjustment.
  • Blinding (single, double, or triple) eliminates performance bias and detection bias, making effect estimates more trustworthy.
  • Results reported to the CONSORT standard are transparent and reproducible, facilitating meta-analysis and systematic review.
  • Accommodates multiple outcome types (continuous, binary, ordinal) and a range of analytic frameworks.
  • Clear temporal sequence: exposure recorded before outcome, establishing directionality and causality more convincingly than cross-sectional designs.
  • Multiple outcomes: single exposure can be related to many outcomes, economical for broad hypothesis generation.
  • Incidence and absolute risk: directly estimates risk in exposed and unexposed groups, enabling calculation of number needed to treat.
  • Avoids outcome-dependent selection: cohort defined by exposure, not outcome, preventing selection bias inherent in case-control studies.
  • Incidence rates: person-time denominators account for variable follow-up, allowing comparison across studies with different follow-up durations.
Limitations
  • Reference standard definition: 'gold standard' may not be perfect or universally agreed. If reference standard is flawed, whole study is flawed.
  • Spectrum bias: if study population does not represent clinical spectrum of disease (e.g., only severe cases, missing mild cases), accuracy estimates do not generalize.
  • Threshold selection: for continuous tests (e.g., biomarker levels), choice of cutoff determines sensitivity/specificity trade-off. Different thresholds yield different accuracy. Study must test range of thresholds or justify single cutoff.
  • Imperfect blinding: if test operator knows disease status, or reference standard assessor knows index test result, bias inflates accuracy.
  • Publication bias: negative or inconclusive accuracy studies may not be published, inflating published accuracy estimates.
  • Randomization is not always ethically or practically feasible — you cannot randomize people to harmful exposures or rare disease states.
  • High cost and logistical complexity; dropout and non-adherence can erode the benefits of randomization if not handled through ITT analysis.
  • External validity (generalizability) may be limited if eligibility criteria exclude the populations in which the intervention will ultimately be deployed.
  • With very small samples (fewer than 10 per group) the trial provides unreliable results; a case study design is preferable in that situation.
  • Time and cost: following cohorts for years or decades requires sustained funding, staff, and infrastructure; attrition (loss to follow-up) is common.
  • Loss to follow-up bias: if participants lost are systematically different on exposure or confounders, RR estimates become biased. Requires high follow-up rates (>80%).
  • Rare outcomes: if outcome incidence is very low, huge cohorts are needed, increasing cost. Case-control may be more efficient.
  • Confounding: unmeasured or poorly measured confounders can bias RR. Adjustment is limited to measured variables.
  • Cohort effects: participants enrolled over time may differ in baseline characteristics, prognosis, or secular trends, affecting comparability.
Common pitfalls
  • Using disease prevalence from study population when applying to different clinical populations: predictive values (PPV, NPV) depend on disease prevalence. A test with high PPV in a high-prevalence population may have low PPV in low-prevalence populations. Use LR instead, which is prevalence-independent.
  • Assuming index test is independent of reference standard: if both tests measure the same biomarker, accuracy is artificially inflated due to correlation, not true independent detection. Use genuinely independent tests.
  • Small sample sizes: studies with <100 participants often have very wide confidence intervals, insufficient precision. Aim for ≥100 diseased and 100 non-diseased.
  • Ignoring spectrum bias: including only severe disease cases or only mild cases biases sensitivity or specificity. Enroll representative cases across disease spectrum.
  • Over-interpreting sensitivity/specificity: without knowing prevalence, these alone don't tell you clinical utility. Report LR and predictive values; use likelihood nomograms to translate test results to clinical probability.
  • Analysing only completers (per-protocol without ITT) inflates apparent efficacy and undermines the intent of randomization.
  • Failing to conduct a power analysis before the trial begins, resulting in an underpowered study that cannot detect clinically meaningful effects.
  • Omitting the allocation concealment mechanism, which allows selection bias to creep back in even after randomization.
  • Reporting only the p-value without the effect size and confidence interval, obscuring clinical meaningfulness.
  • Using simple randomization in small trials without stratification, risking chance imbalance on key prognostic variables.
  • Inadequate follow-up documentation: missing or vague outcome dates, or inconsistent outcome definitions across participants, invalidates temporal inference.
  • Ignoring loss to follow-up: discarding lost participants without analysis inflates selection bias. Use intention-to-treat logic; perform sensitivity analyses under different missing-data mechanisms.
  • Unmeasured confounding: an unmeasured factor (e.g., genetic susceptibility) may explain the exposure-outcome association. Always discuss potential unmeasured confounders.
  • Post-hoc stratification: dividing the cohort into subgroups after seeing results leads to multiple comparisons, false discoveries, and overfitting. Pre-specify subgroup hypotheses.
  • Assuming linearity: in dose-response analyses, assuming a linear relationship between exposure level and outcome can mask threshold or plateau effects. Plot data and test non-linear fits.
Introduced by
  • Bossuyt, Reitsma, and STARD group (2003); clinical epidemiology pioneers
  • James Lind (early precursor, 1747); modern formulation: Austin Bradford Hill & Medical Research Council (1948)
  • Donald Acheson, Olli Miettinen, and others in modern epidemiology
Year
  • 2003-2015
  • 1948
  • 1970s-1980s