External Validation of Simulation-Based Assessments With Other Performance Measures of Third-Year Anesthesiology Residents

Mudumbai, Seshadri C.; Gaba, David M.; Boulet, John R.; Howard, Steven K.; Davies, Marc

doi:10.1097/sih.0b013e31823d018a

Cited by 34 publications

(23 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of these studies showed positive results regarding the reliability and validity of simulation-based assessment when compared to traditional assessment methods. [333435] On the other hand, few studies showed some variability in reliability, especially on evaluating the behavioral aspects. The validity of simulation-based assessments to differentiate a large group of trainees based on clinical experience or training was confirmed in a previous study.…”

Section: Simulation-based Assessmentmentioning

confidence: 99%

Integration of simulation in postgraduate studies in Saudi Arabia: The current practice in anesthesia training program

et al. 2017

View full text Add to dashboard Cite

The educational programs in the Saudi Commission for Health Specialties are developing rapidly in the fields of technical development. Such development is witnessed, particularly in the scientific areas related to what is commonly known as evidence-based medicine. This review highlights the critical need and importance of integrating simulation into anesthesia training and assessment. Furthermore, it describes the current utilization of simulation in anesthesia and critical care assessment process.

show abstract

Section: Simulation-based Assessmentmentioning

confidence: 99%

Integration of simulation in postgraduate studies in Saudi Arabia: The current practice in anesthesia training program

et al. 2017

View full text Add to dashboard Cite

show abstract

“…At the end of a nursing program, the students' performance should reflect the expected competencies of new graduates. Simulations for summative assessment can be short, approximately 3 to 5 minutes, such as when assessing procedures and technical skills, or long, approximately 30 minutes, when assessing behaviors and nontechnical skills such as communication and teamwork (Mudumbai, Gaba, Boulet, Howard, & Davies, 2012). Longer scenarios allow the faculty to create a clinical situation that is complex enough to evaluate students' higher level skills, such as managing a changing patient's condition and setting priorities.…”

Section: Designing Appropriate Simulationsmentioning

confidence: 99%

Summative Simulated-Based Assessment in Nursing Programs

Oermann¹,

Kardong-Edgren²,

Rizzolo³

2016

J Nurs Educ

View full text Add to dashboard Cite

Background: Summative simulated-based assessments are intended to determine students' competence in practice. These assessments need to be carefully designed and implemented especially when the results are used to make high-stakes decisions. Method: Critical steps need to be followed to design simulations for summative assessment, ensure the validity of the assessment and reliability of the ratings, and train evaluators. Results: Guidelines for using simulation for assessment in nursing are suggested. The guidelines are based on the literature and the authors' experiences from a project that examined the feasibility of using simulation for determining students' competence in performance at the end of the nursing program. Conclusion: Summative simulation-based assessments need to be valid, measuring the knowledge and skills they are intended to, and reliable, with results being reproduced by different evaluators and by the same evaluator at another time. [J Nurs Educ. 2016;55(6):323-328.]

show abstract

“…[6] The American Board of Anesthesiology (ABA) has mandated simulated-based training as a part of the practice performance improvement and assessment component for the maintenance of certification in anesthesiology (MOCA). [22] The Accreditation Council for Graduate Medical Education (ACGME) proposed that specific outcomes should assess competency and improve education and performance feedback. [4] 2.…”

Section: Review Of the Literaturementioning

confidence: 99%

Testing an objective structured assessment technical skills tool: A pilot study

Dixon¹,

Burns²

2015

JNEP

View full text Add to dashboard Cite

Purpose: The purpose of this prospective pilot study was to test the feasibility, reliability and validity of an adapted version of an objective structured assessment technical skills (OSATS) tool using a criterion-reference rating scale while evaluating student registered nurse anesthetists (SRNAs) performing mask ventilation, laryngeal mask airway (LMA) insertion, and endotracheal intubation. Methods: Prior to testing, three OSATS evaluation tools were developed with nurse anesthesia faculty input. Six facultyevaluators pilot tested an adapted version of three OSATS tools for feasibility and validity with 23 SRNAs performing mask ventilation, LMA insertion, and endotracheal intubation with an airway management trainer. SRNAs had 15 minutes; five-minute each to perform mask ventilation, insert an LMA, and perform endotracheal intubation. Paired observations from six facultyevaluators determined inter-rater reliability. Separate survey questionnaires were created for faculty-evaluators and SRNAs; using three four-point Likert-type rating scales to measure feasibility and validity. Results: Descriptive statistics revealed that faculty-evaluators' found the OSATs tools to be feasible based on six out of seven somewhat favorable to strongly favorable responses. The Kendall's Coefficient of Concordance W showed statistical significance for mask ventilation and endotracheal intubation (p-value of .041 and .036). Six faculty-evaluators found the OSATs tools to have face and content validity based on six out of seven somewhat favorable responses. Twenty-three SRNAs found the OSATS tools to have face validity based on somewhat strongly favorable responses. Conclusions: An adapted version of OSATS tool for evaluating SRNAs performing mask ventilation, LMA insertion, and endotracheal intubation demonstrated feasibility, reliability, and validity. The inter-rater values obtained from Kendall's Coefficient of Concordance W in this project correlated with the Cook and Beckman report.

show abstract

External Validation of Simulation-Based Assessments With Other Performance Measures of Third-Year Anesthesiology Residents

Abstract: The associations between simulator performance, both for technical and nontechnical skills, and other markers of ability provide some evidence to support the validity of simulation-based assessment scores. Replication studies with larger numbers of residents are warranted.

Cited by 34 publications

References 17 publications

Integration of simulation in postgraduate studies in Saudi Arabia: The current practice in anesthesia training program

Integration of simulation in postgraduate studies in Saudi Arabia: The current practice in anesthesia training program

Summative Simulated-Based Assessment in Nursing Programs

Testing an objective structured assessment technical skills tool: A pilot study

Contact Info

Product

Resources

About