Skip to content
Back to search results

Solomon Four-Group Design — Controlling Pretest Sensitization

Introduced by
Richard L. Solomon
Year
1949
Type
True experimental design
Family
Experimental design
Font size

What does it do?

The Solomon Four-Group Design extends the classic pretest-posttest control-group design by adding two groups that receive no pretest, enabling researchers to detect whether the pretest itself alters participants' responses to the treatment.

Introduced by Richard L. Solomon in 1949, it remains the gold standard for isolating the independent effect of a pretest and for obtaining unbiased estimates of treatment efficacy.

The intuition behind the method

Suppose you are testing a new mindfulness programme. You administer a baseline stress questionnaire to half your participants before the programme begins. Could filling out that questionnaire itself make people more self-aware, and therefore appear more improved at follow-up — even without the programme? The Solomon design answers this directly by running the experiment simultaneously in four groups: two groups get the pretest and two do not, and within each pair one group receives the treatment. By comparing all four posttest means you can cleanly separate what the treatment did from what the pretest did.

History

Richard L. Solomon introduced the design in a 1949 Psychological Bulletin paper as an explicit extension of the classic Campbell-style pretest-posttest control group design, motivated by studies in learning and attitude change where he observed that baseline testing altered participants' subsequent responses. Donald Campbell and Julian Stanley codified it alongside other experimental designs in their 1963 landmark monograph, cementing its place in the research methods canon. Despite its logical elegance, the design has been used sparingly in practice because of its sample size demands, yet it remains the textbook example of how to isolate measurement reactivity in experimental research.

When to use it

Use the Solomon Four-Group Design when you suspect that completing a pretest might change how participants respond to the intervention — common in attitude-change studies, educational research, health behavior interventions, and any domain where answering baseline questions could raise awareness or prime responses. It is most valuable when sensitization is theoretically plausible and the cost of doubling the sample is acceptable. Avoid it when the research budget allows only a small total sample (each of the four groups must be large enough to detect effects independently), when the phenomenon of interest is immune to measurement reactivity, or when pretest data are essential for individual-level tracking — in those cases a standard pretest-posttest design with ANCOVA is more practical.

Limitations and considerations

  • Requires roughly twice the sample size of a standard pretest-posttest design, making it expensive and logistically demanding.
  • Statistical analysis is complex relative to simple pre-post comparisons; ANCOVA on the subset with pretest data requires careful handling of missing covariates for Groups 3 and 4.
  • Does not control for history or maturation effects beyond what randomization provides; these threats apply to all four groups equally.
  • Rarely used in practice because the sensitization assumption is often untested rather than theoretically motivated, and the sample cost is high.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Uniquely identifies and quantifies the pretest sensitization (testing) effect, which simpler designs confound with the treatment effect.
  • Produces the most internally valid estimate of treatment efficacy by combining evidence from both pretested and unpretested groups.
  • Supports a full 2 × 2 factorial ANOVA, yielding main effects for treatment and pretesting as well as their interaction.
  • Random assignment to all four groups supports causal inference for both treatment and pretest effects simultaneously.
  • Provides a built-in replication: the treatment effect can be estimated twice (from Groups 1 vs. 2 and from Groups 3 vs. 4) and the two estimates should converge if the design assumptions hold.

Applications

  • Educational interventions where a pre-course knowledge test might itself prompt learning before the programme begins.
  • Attitude-change and persuasion experiments where completing a pre-measure sensitizes participants to the message.
  • Health behavior programmes (e.g., smoking cessation, dietary change) where baseline questionnaires raise awareness of the target behavior.
  • Psychological training studies examining skill acquisition, where practice effects from pretesting are theoretically plausible.
  • Organizational training and development research where pre-assessment might cue employees to the program objectives.

How it works

How it works

1

Participants are randomly assigned to four groups of roughly equal size: (1) pretest + treatment + posttest, (2) pretest + control + posttest, (3) no pretest + treatment + posttest, and (4) no pretest + control + posttest. Random assignment is essential; without it the design loses its causal interpretation.

2

Groups 1 and 2 complete the pretest measure before any intervention begins. Groups 3 and 4 receive no pretest. The pretest data for Groups 1 and 2 serve both as a baseline covariate and as the basis for estimating the pretest sensitization effect.

3

Groups 1 and 3 receive the experimental treatment; Groups 2 and 4 serve as controls and receive no treatment or an equivalent placebo condition. All other aspects of the study environment are held constant across the four groups.

4

All four groups complete the same outcome measure at the same time point after the intervention period. The posttest is identical across groups so that any mean differences are attributable to treatment and/or pretest effects rather than to measurement differences.

5

Posttest means are compared across all four groups using a 2 × 2 factorial ANOVA (treatment × pretest as factors), or equivalently via ANCOVA using the pretest as a covariate for Groups 1 and 2. A significant Treatment × Pretest interaction indicates sensitization. Main and interaction effects together yield estimates of net treatment effect free from pretest contamination.

Common pitfalls

  • Treating the four groups as two separate 2-group experiments rather than analyzing all posttest means jointly — this wastes power and misses the interaction effect that is the design's main contribution.
  • Failing to verify that random assignment produced balanced groups; even with randomization, checking baseline equivalence across all four groups (for the two pretested groups) and demographic equivalence for all four is important.
  • Including a pretest that is so similar to the intervention content that it constitutes partial treatment rather than mere measurement — this inflates the apparent sensitization effect and distorts estimates.
  • Under-powering individual groups: because the overall sample is divided into four cells, each cell must independently be large enough to contribute to detecting interaction effects; small per-cell n is a common and serious error.

Frequently asked questions

How do I handle the missing pretest scores for Groups 3 and 4 in the analysis?

The recommended approach is a 2 × 2 factorial ANOVA on the four posttest means, treating treatment condition and pretest presence as the two factors. This uses all data without requiring pretest scores for all groups. Alternatively, for Groups 1 and 2 you can run ANCOVA with the pretest as a covariate and compare the ANCOVA-adjusted treatment effect to the unadjusted one from Groups 3 and 4. The two estimates should be close if sensitization is absent.

How large does each group need to be?

Each of the four cells should be powered to detect the expected effect size for the key comparison. Because the interaction (Treatment × Pretest) is typically the test of greatest interest and interaction effects require larger samples than main effects, plan for at least 30–50 participants per cell in most social science contexts, or use an a priori power analysis with your expected effect size and a power of 0.80.

What if the interaction term is not significant?

A non-significant Treatment × Pretest interaction is a useful finding: it indicates that pretesting did not sensitize participants, and the simpler pretest-posttest control-group design would have been adequate. In that case, pool the treatment main effect across both pretest conditions for your primary conclusion.

Is the Solomon design ever used in clinical trials?

Rarely. Clinical trials typically prioritize blinding and cannot always withhold baseline assessments ethically or practically. The design is more common in social, educational, and psychological research where the sensitization threat is more pronounced and withholding a pretest raises no ethical concerns.

What distinguishes the Solomon design from a 2 × 2 factorial experiment?

Structurally it is a 2 × 2 factorial (treatment × pretest), but the 'pretest' factor is not a manipulated treatment variable — it is a design feature whose presence or absence is assigned to control a nuisance. The Solomon design is therefore better understood as a method for purifying causal inference about the main treatment than as a factorial study of two independent factors.

Related methods

Sources

  1. Solomon, R. L. (1949). An extension of control group design. Psychological Bulletin, 46(2), 137–150.

    Open DOI reference
    Copied.
  2. Campbell, D. T., & Stanley, J. C. (1963). Experimental and Quasi-Experimental Designs for Research. Rand McNally.

    ISBN: 978-0395307878
    Copied.