Skip to content
Back to search results

Delphi Technique — Iterative Expert Consensus Method

Introduced by
Norman Dalkey and Olaf Helmer (RAND Corporation)
Year
1950s–1963
Type
Iterative expert consensus technique
Family
Data collection
Font size

What does it do?

The Delphi technique is a structured, multi-round data collection method that harvests and refines expert opinion through iterative questionnaires and controlled feedback.

Developed at RAND Corporation in the 1950s, it is designed to converge a dispersed expert panel toward a reliable consensus on complex, uncertain, or future-oriented questions — without the conformity pressures of face-to-face group discussion.

The intuition behind the method

Imagine gathering thirty specialists from different continents to agree on clinical practice guidelines, but you cannot get them in the same room and you do not want the most vocal expert to dominate. Delphi solves this by sending questions in rounds: experts answer anonymously, see a statistical summary of everyone else's answers, then revise their own positions. Because feedback is anonymous and aggregated — typically a median or interquartile range — each expert can see how they sit relative to the group and reconsider without social pressure. Rounds continue until responses stabilise, producing a defensible, group-level expert judgment rather than a single individual's opinion.

History

The Delphi method was developed by Norman Dalkey and Olaf Helmer at the RAND Corporation in the early 1950s as a classified forecasting tool, first published openly in a 1963 Management Science article. The name references the Oracle at Delphi, reflecting its original purpose of producing expert forecasts. Harold Linstone and Murray Turoff's 1975 edited volume The Delphi Method: Techniques and Applications established it as a mainstream research tool. From the 1980s onward the technique was widely adopted in health sciences, education, and policy research. Online variants emerged in the 2000s, and the real-time Delphi and e-Delphi formats now dominate practice.

When to use it

Use the Delphi technique when you need a structured expert judgment on a topic where no empirical data exist or where existing evidence is insufficient, contested, or highly uncertain — for example, forecasting technology trends, developing professional standards, or prioritising research agendas. It is well suited when experts are geographically dispersed, when anonymity is needed to prevent authority bias or groupthink, and when several rounds of revision are feasible within your timeline. Do not use Delphi when a quick decision is needed (three or more rounds take weeks to months), when the topic requires stakeholder input from non-experts rather than specialist knowledge, or when the sample of qualified experts is too small (fewer than seven) to yield a meaningful distribution.

Limitations and considerations

  • Time-consuming: two to four rounds with synthesis between each round typically spans six weeks to several months.
  • Response attrition across rounds can shrink an already small panel to an unrepresentative remainder.
  • Consensus may reflect anchoring or convergence pressure rather than genuine expert agreement, especially when feedback framing is biased.
  • Quality of findings depends heavily on the expertise and representativeness of the panellists — a poorly selected panel produces misleading consensus.
  • The method elicits expert opinion, not empirical data; consensus is not the same as correctness.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Anonymity neutralises social pressure and authority bias, enabling genuine revision of positions across rounds.
  • Geographically dispersed experts can participate without travel, broadening the talent pool.
  • Controlled feedback loop produces measurable convergence — consensus can be quantified rather than assumed.
  • Flexible format accommodates quantitative ratings, rankings, and qualitative justifications within the same instrument.
  • Transparent reporting of dissent: minority views and their rationales are preserved alongside the consensus finding.

Applications

  • Developing clinical practice guidelines and consensus-based diagnostic criteria in medicine and nursing.
  • Prioritising research agendas in fields where empirical evidence is sparse or conflicting.
  • Forecasting technological developments, market trends, or policy impacts over a 5–20 year horizon.
  • Identifying and weighting criteria for evaluation frameworks, accreditation standards, or curriculum design.
  • Reaching cross-disciplinary agreement on terminology, classification systems, or competency frameworks.
  • Policy Delphi: deliberately surfacing and structuring diverse expert positions to inform public decision-making rather than forcing a single consensus.

How it works

How it works

1

A purposive sample of experts — typically 10–30, though panels up to 100 are used in large policy studies — is recruited based on demonstrated knowledge of the topic. Anonymity among panellists is maintained throughout; experts know only aggregate statistics about each other's responses, not individual attributions.

2

The first round is usually open-ended: experts are asked to list key factors, criteria, or risks related to the focal question. Their responses are qualitatively analysed and distilled into a structured item list (statements, criteria, ratings scales) that forms the questionnaire used in subsequent rounds.

3

After each round the facilitating team quantifies responses — computing medians, means, interquartile ranges, and percentage agreement — and prepares a controlled feedback report. This report shows each expert their own prior response alongside the group distribution, allowing direct comparison without revealing who said what.

4

Experts receive the feedback report and complete the revised questionnaire, now able to maintain or adjust their position in light of group consensus. They are typically invited to provide reasons if their response falls outside the interquartile range. Usually two to four rounds are conducted; rounds cease when predefined stopping criteria (stability, consensus threshold, or round limit) are met.

5

Consensus is assessed quantitatively — common thresholds include at least 70–80% agreement within a defined response category, or an interquartile range of one scale point or less. The final-round distribution of expert responses, medians, and any dissenting rationales constitute the primary output and are reported alongside the consensus level.

Common pitfalls

  • Neglecting attrition management: failing to send reminders, shorten later-round instruments, or replace drop-outs can leave the final round with too few respondents to claim consensus.
  • Premature closure: stopping after only one round because responses look similar is methodologically insufficient — at least two rounds are required to demonstrate stability.
  • Biased feedback framing: presenting the group median in a way that implicitly pressures outliers to conform undermines anonymity and corrupts the controlled-feedback mechanism.
  • Vague stopping criteria: consensus thresholds (e.g., percentage agreement, IQR cutoff) must be specified before data collection, not chosen after seeing the results.
  • Confusing Delphi with a survey: a one-shot expert questionnaire is not Delphi — the iterative, feedback-driven structure is the defining feature.

Frequently asked questions

How many rounds does a Delphi study need?

A minimum of two rounds is required to demonstrate that responses have been revised in light of group feedback and that stability has been tested. Most studies use two to four rounds; three is the most common. Stopping is justified when a predefined stability or consensus criterion is met, not simply when time runs out.

How large should the expert panel be?

There is no universal rule, but 10–30 panellists is the most widely cited workable range. The key requirement is that panellists collectively cover the domain of expertise relevant to the question. Panels smaller than about 7–10 are too narrow to produce a stable group distribution; very large panels (over 100) complicate synthesis and increase attrition risk.

What distinguishes Delphi from a regular expert survey?

A regular expert survey collects responses once. Delphi is iterative: experts see anonymised feedback about the group distribution after each round and can revise their answers. This controlled feedback loop, combined with anonymity, is the defining feature that separates Delphi from a single-round expert questionnaire.

How is consensus defined and reported?

Consensus thresholds should be set before data collection. Common criteria include: at least 70–80% of responses falling in a defined category, an interquartile range of one point or less on a Likert scale, or a coefficient of variation below a chosen cutoff. The final report should state the threshold used, the level achieved for each item, and the median together with the interquartile range.

Can Delphi be conducted online?

Yes. E-Delphi and real-time Delphi platforms (such as SurveyMonkey, Qualtrics, or dedicated tools like the RAND/UCLA Appropriateness Method software) allow asynchronous participation and automated feedback calculation. Online delivery reduces logistical burden and can shorten turnaround between rounds, but requires careful attention to data security and participant authentication.

Related methods

Sources

  1. Dalkey, N., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458–467.

    Open DOI reference
    Copied.
  2. Linstone, H. A., & Turoff, M. (Eds.). (1975). The Delphi Method: Techniques and Applications. Addison-Wesley.

    Open publication reference
    Copied.