Validity and reliability of an objective structured assessment tool for performance of ultrasound-guided regional anaesthesia

Shafqat, Atif; Rafi, Malik; Thanawala, V.; Bedforth, N.; Hardman, Jonathan G.; McCahon, R.

doi:10.1016/j.bja.2018.06.014

Cited by 6 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our study is the first to validate steps and errors for the measurement of translational performance from cadavers to patients, to validate eye tracking technology as a quantitative measure of regional anaesthesia performance, and to inform the debate surrounding the best measure of performance reliability. 42 Our metrics will enable us to measure the translation of performance from simulators to patients, and ask important questions about the impact of simulation in regional anaesthesia on clinical performance. This study is the first of three sequential studies.…”

Section: Wider Impact Of Researchmentioning

confidence: 99%

Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients

McLeod

McKendrick

Taylor

et al. 2019

British Journal of Anaesthesia

View full text Add to dashboard Cite

Background: Our goal is to develop metrics that quantify the translation of performance from cadavers to patients. Our primary objective was to develop steps and error checklists from a Delphi questionnaire. Our second objective was to show that our test scores were valid and reliable. Methods: Sixteen UK experts identified 15 steps conducive to good performance and 15 errors to be avoided during interscalene block on the soft-embalmed cadaver and patients. Thereafter, six experts and six novices were trained, and then tested. Training consisted of psychometric assessment, an anatomy tutorial, volunteer scanning, and ultrasoundguided needle insertion on a pork phantom and on a soft-embalmed cadaver. For testing, participants conducted a single interscalene block on a dedicated soft-embalmed cadaver whilst wearing eye tracking glasses. Results: We developed a 15-step checklist and a 15-error checklist. The internal consistency of our steps measures were 0.83 (95% confidence interval [CI]: 0.78e0.89) and 0.90 (95% CI: 0.87e0.93) for our error measures. The experts completed more steps (mean difference: 3.2 [95% CI: 1.5e4.8]; P<0.001), had less errors (mean difference: 4.9 [95% CI: 3.5e6.3]; P<0.001), had better global rating scores (mean difference: 6.8 [95% CI: 3.6e10.0]; P<0.001), and more eye-gaze fixations (median of differences: 128 [95% CI: 0e288]; P¼0.048). Fixation count correlated negatively with steps (r¼e0.60; P¼0.04) and with errors (r¼0.64; P¼0.03). Conclusions: Our tests to quantify ultrasound-guided interscalene nerve block training and performance were valid and reliable.

show abstract

Section: Wider Impact Of Researchmentioning

confidence: 99%

Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients

McLeod

McKendrick

Taylor

et al. 2019

British Journal of Anaesthesia

View full text Add to dashboard Cite

show abstract

“…However, the expertsʼ consensus is on using a combination of the GRS and validated procedure-specific checklists as the gold standard for procedure-specific assessments. 13,14) The GRS overestimates respect for tissue and quality of products, whereas the TSRS does so for all other domains. We assumed a difference of <15% as an acceptable range; overall and all task-specific domains estimates fell within this range, except for knotting and suturing that exceeded this range.…”

Section: Discussionmentioning

confidence: 97%

Validation of Task-Specific Rating Scale for Open Balloon Catheter Arterial Embolectomy: An Assessor-Blinded Quasi-Experimental Pilot Study

Siddiqui

Hashmi

Naz

et al. 2022

Annals of Vascular Diseases

View full text Add to dashboard Cite

Objective: To develop and validate a task-specific rating scale (TSRS) by comparing with the Global Rating Scale (GRS) for the evaluation of brachial artery embolectomy (BAE). Methods: Participants were divided into expert and novice groups who were oriented on the locally developed simulator model. The following day, an embolectomy procedure was performed independently by the participants and graded by two independent assessors using the GRS and TSRS. Validity was evaluated using Pearsonʼs correlation coefficient (r), reliability by the interclass correlation coefficient (ICC), and agreement by Bland-Altman plots. A p-value <0.05 was considered significant. Results: Thirty-two participants were enrolled in this study. The overall TSRS was found to be a valid assessment tool (r=0.82; 95% confidence interval [CI]: 0.66, 0.91; p<0.001). Domain-specific analyses showed a moderate positive association between all domains (p<0.05), except for instrument handling (r=0.09; 95%CI: −0.27, 0.42; p=0.642). The ICC for overall scores showed excellent reliability for both instruments, GRS and TSRS, with values of 0.97 and 0.92, respectively. Conclusion:The TSRS was found to be a valid and reliable assessment tool for BAE; however, for some domains, such as instrument handling and time and motion, it has limited reliability.

show abstract

“…This ensures that nursing assessments are less subjective, increases the certainty of nurses and improves decision-making about patient care. [ 13 ]. However, these instruments are used independently, although they share constructs, dimensions and items related to mobility, hygiene or feeding, resulting in redundant assessments of care needs and problems associated with hospitalization [ 6 , 14 ].…”

Section: Introductionmentioning

confidence: 99%

Development and Validation of a Meta-Instrument for the Assessment of Functional Capacity, the Risk of Falls and Pressure Injuries in Adult Hospitalization Units (VALENF Instrument) (Part II)

Luna-Aleixós

Llagostera-Reverter

Castelló-Benavent

et al. 2023

IJERPH

View full text Add to dashboard Cite

The nursing assessment is the first step of the nursing process and fundamental to detecting patients’ care needs and at-risk situations. This article presents the psychometric properties of the VALENF Instrument, a recently developed meta-instrument with only seven items that integrates the assessment of functional capacity, risk of pressure injuries and risk of falls with a more parsimonious approach to nursing assessment in adult hospitalization units. A cross-sectional study based on recorded data in a sample of 1352 nursing assessments was conducted. Sociodemographic variables and assessments of the Barthel, Braden and Downton instruments were included at the time of admission through the electronic health history. Thus, the VALENF Instrument obtained high content validity (S-CVI = 0.961), construct validity (RMSEA = 0.072; TLI = 0.968) and internal consistency (Ω = 0.864). However, the inter-observer reliability results were not conclusive, with Kappa values ranging between 0.213 and 0.902 points. The VALENF Instrument has adequate psychometric properties (content validity, construct validity, internal consistency and inter-observer reliability) for assessing the level of functional capacity, risk of pressure injuries and risk of falls. Future studies are necessary to establish its diagnostic accuracy.

show abstract

Validity and reliability of an objective structured assessment tool for performance of ultrasound-guided regional anaesthesia

Abstract: Both assessments differentiated between individuals who had performed fewer (≤30) and many (>100) blocks in the preceding year, supporting construct validity. It also established concurrent validity and overall reliability. We recommend that both tools can be used in UGRA assessment.

Cited by 6 publications

References 23 publications

Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients

Validity and reliability of metrics for translation of regional anaesthesia performance from cadavers to patients

Validation of Task-Specific Rating Scale for Open Balloon Catheter Arterial Embolectomy: An Assessor-Blinded Quasi-Experimental Pilot Study

Development and Validation of a Meta-Instrument for the Assessment of Functional Capacity, the Risk of Falls and Pressure Injuries in Adult Hospitalization Units (VALENF Instrument) (Part II)

Contact Info

Product

Resources

About