Stability of response characteristics of a Delphi panel: application of bootstrap data expansion

Akins, Ralitsa; Tolson, Homer; Cole, Bryan R.

doi:10.4016/4698.01

Cited by 154 publications

(212 citation statements)

References 0 publications

Supporting

Mentioning

204

Contrasting

Unclassified

Order By: Relevance

“…Furthermore, such a small size may not affect the validity of our work. A study by Akins and colleagues considered the panel size used in the Delphi technique and concluded: 'Experts who have similar training and general understanding in the field of interest allow for effective and reliable utilisation of a small sample from a limited number of experts' [35].…”

Section: Cognitive Aid Descriptionmentioning

confidence: 99%

Cognitive Aids in Medicine Assessment Tool (CMAT): preliminary validation of a novel tool for the assessment of emergency cognitive aids

et al. 2015

View full text Add to dashboard Cite

Summary Applying human factors principles to the design of clinical emergency guidelines is important. The UK Civil Aviation Authority uses a Checklist Assessment Tool for evaluating the content and usability of emergency drills before introduction into service on aircraft. We hypothesised that this model could be used to develop a generic medical tool. A three‐stage modified Delphi process was used to adapt the above tool for use in designing medical emergency guidelines. The resulting Cognitive aids in Medicine Assessment Tool was then used to score and rank seven published difficult airway guidelines; the scores were used to assess its validity and reliability. Pearson's rank coefficient between these scores and scores from independent assessors was 0.89 (p = 0.007). Internal consistency, as assessed by Cronbach's alpha, was 0.74, 0.96 and 0.72 for the tool's three constituent domains of physical characteristics, content and layout/format, respectively. Inter‐rater reliability, as assessed by Cohen's kappa, ranged from 0.33 to 0.72. The adoption of our tool has the potential to improve the usability of medical emergency guidelines.

show abstract

Section: Cognitive Aid Descriptionmentioning

confidence: 99%

Cognitive Aids in Medicine Assessment Tool (CMAT): preliminary validation of a novel tool for the assessment of emergency cognitive aids

et al. 2015

View full text Add to dashboard Cite

show abstract

“…The number of participants could have affected the potential for ideas as well as the amount of data analysed. However, no agreement on the panel size for Delphi studies exists, and neither do recommendations or unequivocal definitions of small or large samples 17 . Many published Delphi studies use panels consisting of 10-100 or more panellists 17 .…”

Section: Strengths and Weaknessesmentioning

confidence: 99%

“…However, no agreement on the panel size for Delphi studies exists, and neither do recommendations or unequivocal definitions of small or large samples 17 . Many published Delphi studies use panels consisting of 10-100 or more panellists 17 . Official consensus was obtained regarding all 61 definitions, but not all experts fully agreed with all final definitions.…”

Section: Strengths and Weaknessesmentioning

confidence: 99%

Semantics in active surveillance for men with localized prostate cancer — results of a modified Delphi consensus procedure

et al. 2017

View full text Add to dashboard Cite

| Active surveillance (AS) is broadly described as a management option for men with low-risk prostate cancer, but semantic heterogeneity exists in both the literature and in guidelines. To address this issue, a panel of leading prostate cancer specialists in the field of AS participated in a consensus-forming project using a modified Delphi method to reach international consensus on definitions of terms related to this management option. An iterative three-round sequence of online questionnaires designed to address 61 individual items was completed by each panel member. Consensus was considered to be reached if ≥70% of the experts agreed on a definition. To facilitate a common understanding among all experts involved and resolve potential ambiguities, a face-to-face consensus meeting was held between Delphi survey rounds two and three. Convenience sampling was used to construct the panel of experts. In total, 12 experts from Australia, France, Finland, Italy, the Netherlands, Japan, the UK, Canada and the USA participated. By the end of the Delphi process, formal consensus was achieved for 100% (n = 61) of the terms and a glossary was then developed. Agreement between international experts has been reached on relevant terms and subsequent definitions regarding AS for patients with localized prostate cancer. This standard terminology could support multidisciplinary communication, reduce the extent of variations in clinical practice and optimize clinical decision making.312 | MAY 2017 | VOLUME 14 www.nature.com/nrurol CONSENSUS STATEMENT © 2 0 1 7 M a c m i l l a n P u b l i s h e r s L i m i t e d , p a r t o f S p r i n g e r N a t u r e . A l l r i g h t s r e s e r v e d .degree of monitoring than those on AS might receive, and in which palliative treatment is generally instituted if metastases or local symptoms develop. Interpreting and comparing research results is difficult owing to the nonstandardized use of the terms AS and WW and their intended and often mixed treatment objectives (curative or palliative) 5 . This semantic heterogeneity is also reflected in AS guidelines 6 . In these guidelines, AS is primarily recommended for patients with low-risk tumours; however, these guidelines contain various definitions of low-risk prostate cancer, as specified by different combinations of clinical criteria including clinical and pathological characteristics (such as tumour stage, serum PSA levels, biopsy Gleason score, tumour volume and serum PSA density). Furthermore, definitions of disease reclassification and progression differ among published guidelines and multiple criteria for initiation of curative treatment are proposed 6 .Problems resulting from the use of ambiguous language include hindered clinical decision making, particularly in multidisciplinary collaborations, and limited opportunities for research 7 . Moreover, such ambiguity has raised a barrier that hampers exchange of knowledge within and between fundamental domains of research and research groups 8 . An urgent need exists for uniform...

show abstract

“…Sample size: Given the well-defined area of knowledge and the limited number of experts in the field, a sample size of at least 14 experts providing complete responses to the survey was considered adequate in the context of the study (34). Patients were enrolled until reaching saturation (i.e., no new items were obtained), which usually entails between 5 and 25 patients, based on general guidelines for the conduction of qualitative studies using the phenomenological approach (35).…”

Section: 2mentioning

confidence: 99%

Pediatric post-thrombotic syndrome in children: Toward the development of a new diagnostic and evaluative measurement tool

Avila

Brandão

Williams

et al. 2016

Thrombosis Research

View full text Add to dashboard Cite

Objective: Our goal was to conduct the generation and piloting of a discriminative and evaluative tool for pediatric post-thrombotic syndrome.Methods: We followed a formative model for the development of the tool, focusing on the signs/symptoms (items) that define post-thrombotic syndrome. For item generation, pediatric thrombosis experts and diagnosed with extremity post-thrombotic syndrome during childhood nominated items. In the piloting phase, items were cross-sectionally measured in children with limb deep vein thrombosis to examine the performance of the items.Result: Twenty-three experts and 16 subjects listed 34 items, which were then measured in 140 subjects with previous diagnosis of limb deep vein thrombosis (70 upper extremity and 70 lower extremity). The items with strongest correlation with post-thrombotic syndrome severity and largest area under the curve were pain (in older children), paresthesia, and swollen limb for the upper extremity group, and pain (in older children), tired limb, heaviness, tightness and paresthesia for the lower extremity group.Conclusion: Diagnostic properties and correlation with post-thrombotic syndrome severity of the different signs and symptoms varied according to the assessed venous territory.The information gathered in this study will help experts decide which item should be considered for inclusion in the new tool.

show abstract

Stability of response characteristics of a Delphi panel: application of bootstrap data expansion

Cited by 154 publications

References 0 publications

Cognitive Aids in Medicine Assessment Tool (CMAT): preliminary validation of a novel tool for the assessment of emergency cognitive aids

Cognitive Aids in Medicine Assessment Tool (CMAT): preliminary validation of a novel tool for the assessment of emergency cognitive aids

Semantics in active surveillance for men with localized prostate cancer — results of a modified Delphi consensus procedure

Pediatric post-thrombotic syndrome in children: Toward the development of a new diagnostic and evaluative measurement tool

Contact Info

Product

Resources

About