Lessons Learned from a Two-Round Delphi-based Scenario Study

Abstract

The Delphi technique is a suitable methodology for structuring group communication to answer current and prospective research questions within several rounds. The method is used in many disciplines and characterized by anonymity, iteration, controlled feedback, and statistical “group response” (Rowe & Wright, 2001). This technical paper presents practical details and lessons learned from a two-round Delphi-based scenario study in which projections (Delphi statements, questions or hypotheses) were developed with findings from expert interviews and an expert workshop. This Delphi study provides answers to future-related questions for which other research methods are inappropriate. This is depicted as a five-step process, making it easy to follow and replicable, for example to help first-time Delphi-method researchers. In doing so, the authors aim to provide the community with valuable technical insights and guidance for studies applying the Delphi technique both to prospective questions and in other research settings. Conducting a Delphi study can be a slow process with respect to receiving feedback from the panelists. Planning an appropriate period for distributing the questionnaire may produce a higher return rate. A sufficient time buffer should be incorporated into project planning. Projections that create dissent among the panelists may provide valuable results. Data analytics, software programs and online social networks can support researchers, save time and resources, and provide further insights in the process of conducting a Delphi study.

Preparing, conducting, and analyzing Delphi surveys: Cross-disciplinary practices, new directions, and advancements

“…In this context, we provide a comprehensive overview of potential features and recent advancements in all three phases (see Table 1 , [5] , [6] , [17] , [23] , [26] , [28] , [30] , [31] , [37] , [39] , [41] , [46] , [55] , [57] , [67] , [64] , [65] , [79] , [85] ) and therefore complement and substantially extend recent methodic publications such as Schmalz et al. [80] . Thereby, we aim to support the research community in utilizing the Delphi technique for their respective disciplines by following a replicable, but still highly customizable approach.…”

“…Thorough preparation is critical to ensure the validity and accuracy of a Delphi study [45 , 80] . In general, this phase pursues four different goals: (1) Definition of research goals , (2) definition of Delphi format , (3) definition of Delphi statements , and (4) definition of additional questions (see Fig.…”

“…As Schmalz et al. [80] conclude, a thorough literature review is indispensable for a Delphi study. However, this does not necessarily need to be limited to the scientific body of research – particularly in the case of prospective, forecast studies for which existing literature might be scarce.…”

Applying real-time Delphi methods: development of a pain management survey in emergency nursing

“…item consensus and selection) [ 56 ]. Conducting a classic multi-round Delphi study can be a slower process with respect to receiving and analysing feedback, generating the next survey round and determining consensus, and potentially increases the risk of attrition [ 66 ]. A potential benefit of the real-time Delphi is its expediency [ 67 ].…”

Changes in production work when using smart wearables—results of a Delphi-based study

“…Opinions on the degree of diffusion of the scenario #3 (Independent learning for the future; 2nd round: IQR = 50, 3rd round: IQR = 45) and scenario #6 (constant competition; 2nd round: IQR = 45, 3rd round: IQR = 35) differed particularly strongly. As Schmalz et al (2021) point out, it is beneficial not only to focus on the scenarios for which consensus has been reached, but also to take a closer look at the scenarios where there is the greatest disagreement among experts, i.e. the scenarios with the highest IQR.…”