An accessible analytical approach for investigating what happens between the rounds of a Delphi study

Abstract

The Delphi method is used to investigate consensus amongst a panel of experts using repeated rounds of a questionnaire, often in healthcare settings. However, many Delphi studies do not report any investigation into what happens to the stability of consensus or the convergence of agreement between the rounds in the study, which may be of importance. In this paper an accessible analytical approach is outlined using graphical presentations of means and standard deviations to identify what happens between rounds. For Delphi studies where the scale upon which experts are expressing their opinions can be considered to be interval, the mean will represent the group opinion whilst the standard deviation will represent the level of agreement. An example Delphi study from a healthcare setting is used to illustrate the methodology.

Delphi approach to select rare diseases for a European representative survey. The BURQOL-RD study

“…The phenomenon of the agreement between the panel members increasing between the rounds is recognised in literature [43], but whether the increase in agreement is because the feedback has worked in a constructive way to help the experts who are out of line with the consensus to refine their judgments, or whether such experts have just conformed to the majority view is difficult to tell [44]. However, looking at the level of accordance of the final selection with the preestablished criteria, we consider that this methodology has met the project's objective of representativeness of studied RDs.…”

“…A, B and C represent the three chosen (17) Connective tissue disorders (12) Hereditary angioedema (8) Fabry disease (7) Mucopolysaccharidosis (22) Glycogenstoragedisease (21) Duchenne muscular dystrophy (57) Epidermolysis bullosa (44) Osteogenesis imperfecta (15) Rett syndrome (15) Amyotrophic lateral sclerosis (12) Tuberous sclerosis (5) Degenerative ataxias (5) Syringomyelia (2) Usher syndrome (1) Histiocytosis (6) Source: BURQOL-RD. Note: Results from the prioritisation in brackets.…”

Generic Descriptors: a health check

“…This information might have identified why particular descriptors reached a consensus and others did not. Other researchers using the delphi method might be advised to consider these issues (Greatorex & Dexter, 1998). • The definition of consensus which was used meant that not all the descriptors could have reached a consensus.…”

Occupational therapists’ reflection on practice within psychiatric care: A Delphi study

“…These viewpoints are subsequently synthesized and offered back to the group to elicit their opinions. It is accepted that the respondents should be able to modify their opinions regarding the consensus generated in previous rounds (27). When the experts have reached a consensus data collection comes to an end (28).…”