The Delphi method was developed in the United States in the 1950s and 1960s in the military domain. It has been considered particularly useful in helping researchers determine the range of opinions which exist on a particular subject, in investigating issues of policy or clinical relevance and in trying to come to a consensus on controversial issues. The objectives can be roughly divided into those which aim to measure diversity and those which aim to reach consensus.
The Delphi method is a systematic interactive way of gaining opinions/forecasts from a panel of independent experts over 2 or more rounds. It is a type of consensus method.
The 3 best known consensus techniques are:
- Delphi process
- Nominal group technique
- Consensus development conference
Collaborative estimating or forecasting technique that combines independent analysis with maximum use of feedback, for building consensus among experts who interact anonymously. The topic under discussion is circulated (in a series of rounds) among participating experts who comment on it and modify the opinion(s) reached up to that point … and so on until some degree of mutual agreement is reached. Also called delphi forecasting.
The Delphi method is a combination of qualitative and quantitative processes that draws mainly upon the opinions of identified experts to develop theories and projections for the future. A group of experts is drawn from several disciplines and professions. A multiple-round survey system is administered to this group over an extended period of time. The goal of this method is to reach a consensus among the group by the end of this multiple-round questionnaire process. The uniqueness of Delphi lies in its reliability, given the variableness of human opinion, and in its ability to be administered remotely and without direct participant interaction. It is best used for a fairly simple assessment of new products and developments, but it is one of the most complex methodologies available.
Rowe and Wright (1999) characterize the classical Delphi method by four key features:
- Anonymity of Delphi participants: allows the participants to freely express their opinions without undue social pressures to conform from others in the group. Decisions are evaluated on their merit, rather than who has proposed the idea.
- Iteration: allows the participants to refine their views in light of the progress of the group’s work from round to round.
- Controlled feedback: informs the participants of the other participant’s perspectives, and provides the opportunity for Delphi participants to clarify or change their views.
- Statistical aggregation of group response: allows for a quantitative analysis and interpretation of data.
Fig 1:
The following steps outline how to undertake a Delphi study:
- Design the questionnaire
- Invite participants to take part
- Send out first round of questionnaire- a typical question may be-
Example:
Which of the following CLINICAL areas do you think are HIGH PRIORITY for development of an improved evidence base relating to minority ethnic groups and their health needs?
| Clinical Area | PRIORITY for development of an improved evidence base | Comments including any particularly important topics for action. |
| Mental Health | Low 1 2 3 4 5 High 0 (dont know) | |
| Cancer | Low 1 2 3 4 5 High 0 (dont know) | |
| Immunisation/Vaccination | Low 1 2 3 4 5 High 0 (dont know) |
The questionnaire provides space for respondents to raise any other issues relating to the topic. The first round of the questionnaire aims to categorise opinions under common headings.
4.Analyse responses from round 1 questionnaire
5.Prepare the second round questionnaire
6.Send out second round questionnaire – the follow up to the first round question above might be-
Example:
Cancer has been identified as a high priority for developing an evidence base relating to minority ethnic groups. Within this clinical area, what aspects should research focus on?
| Research area | PRIORITY for development of an improved evidence base | Comments including any particularly important topics for action. |
| Identifying risk factors of disease | Low 1 2 3 4 5 High 0 (dont know) | |
| Identifying barriers to access of services | Low 1 2 3 4 5 High 0 (dont know) | |
| Improving the patient experience for minority ethnic groups | Low 1 2 3 4 5 High 0 (dont know) |
In this example, the research areas chosen for the second round questionnaire were identified from the free text column in the round 1 questionnaire. Participants have the chance to suggest further areas of focus in the second round questionnaire. The aim of second round is to score agreement or disagreement with statements from first round.
7.Analyse responses from round 2 questionnaire
8.Design the third round questionnaire. For the third round, the second questionnaire is repeated but incorporates scores from the second questionnaire results. This gives participants a chance to see how the rest of the group prioritised the areas and if the participant then wants to change their opinion on the basis of the group consensus, has the opportunity to do so.
9.Analyse the results of the third round questionnaire for agreement and degree of consensus
10.Report findings
Strengths of the Delphi technique:
- A rapid consensus can be achieved
- Participants do not have to be in the same room together to reach agreement
- Individuals are able to express their own opinions as opposed to “Group think”
- Can include a wide range of expertise
- Relatively low cost to administer and analyse
- There is the potential to gain large quantities of data
Weaknesses of Delphi technique:
- Does not cope well with widely differing opinions or large changes in opinions (paradigm shifts)
- The facilitator’s view may dominate in the analysis
- Differing opinions may not be sufficiently investigated
- Can be time-consuming
- Needs high participant motivation
- Success of the method depends on the quality of the participants