An Adaptation of the RAND/UCLA Modified Delphi Panel Method in the Time of COVID-19

Abstract

The RAND/UCLA modified Delphi panel method is a formal group consensus process that systematically and quantitatively combines expert opinion and evidence by asking panelists to rate, discuss, then re-rate items. The method has been used to develop medical society guidelines, other clinical practice guidelines, disease classification systems, research agendas, and quality improvement interventions. Traditionally, a group of experts meet in person to discuss results of a first-round survey. After the meeting, experts complete a second-round survey used to develop areas of consensus. During the COVID-19 pandemic, this aspect of the method was not possible. As such, we have adapted the method to conduct virtual RAND/UCLA modified Delphi panels. In this study, we present a targeted literature review to describe and summarize the existing evidence on the RAND/UCLA modified Delphi panel method and outline our adaptation for conducting these panels virtually. Transitioning from in-person to virtual meetings was not without challenges, but there have also been unexpected advantages. The method we describe here can be a cost-effective and efficient alternative for researchers and clinicians.

Consensus recommendations for measuring the impact of contraception on the menstrual cycle in contraceptive clinical trials

“…The second objective also included determining what research and evidence are needed to empirically evaluate the recommended improvements. Our consensus-building methodology was an amalgamation of the RAND/UCLA method, the Delphi method, the Modified Rand-Delphi method, the Nominal Group Technique, and the Jandhyala method because no single approach met all our needs [30][31][32][33][34][35]. We present extensive details on the methodology in the Supplementary Material, but briefly, the fully virtual consensus-building process, depicted in Figure 3, included four half-day meetings for all experts to join, recommendation development within five smaller topical working groups into which experts self-selected, and a series of consensus questionnaires around the scope and details of the recommendations.…”

Relative energy deficiency in dance (RED-D): a consensus method approach to REDs in dance

“…Methodological considerations were made using the ‘Clinical Consensus Manual’ 15. The RAND/UCLA Delphi Panel Method was employed due to its ability to develop clinical guidelines using a process whereby experts would be asked to rate, discuss and then re-rate items established through the literature review 16 17. A round of focus group discussions was undertaken, including patient and public involvement (PPI) groups (figure 1).…”

Expert Consensus on the Use of On‐Demand Treatments for OFF Episodes in Parkinson's Disease: A Modified Delphi Panel

“…We used the RAND/UCLA modified Delphi panel method, which systematically and quantitatively combines expert opinion and the latest clinical evidence. 10 , 11 , 12 , 13 We convened a panel of experts who reviewed a summary of clinical evidence on the use of on ‐demand treatments and rated the appropriateness of prescribing on ‐demand treatments across 432 patient scenarios before a meeting. Panelists discussed these ratings during an 8‐hour professionally moderated discussion, and then rated the same scenarios a second time.…”