Reliability and Validity Testing of the Creighton Competency Evaluation Instrument for Use in the NCSBN National Simulation Study

Hayden, Jennifer; Keegan, Mary J.; Kardong-Edgren, Suzan; Smiley, Richard

doi:10.5480/13-1130.1

Cited by 106 publications

(68 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…[4,19] The investigators were able to standardize practical skills for evaluating airway management skills performances for simulation laboratory testing. [2] The 50/50 survey responses by the SRNAs to questions asking whether their skills were assessed objectively or subjectively is unknown but, may be due to a perception that the tools were biased, unrealistic, and did not reflect their previous graded skills performance. The OSATS tool went through several transformations.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Testing an objective structured assessment technical skills tool: A pilot study

Dixon¹,

Burns²

2015

JNEP

View full text Add to dashboard Cite

Purpose: The purpose of this prospective pilot study was to test the feasibility, reliability and validity of an adapted version of an objective structured assessment technical skills (OSATS) tool using a criterion-reference rating scale while evaluating student registered nurse anesthetists (SRNAs) performing mask ventilation, laryngeal mask airway (LMA) insertion, and endotracheal intubation. Methods: Prior to testing, three OSATS evaluation tools were developed with nurse anesthesia faculty input. Six facultyevaluators pilot tested an adapted version of three OSATS tools for feasibility and validity with 23 SRNAs performing mask ventilation, LMA insertion, and endotracheal intubation with an airway management trainer. SRNAs had 15 minutes; five-minute each to perform mask ventilation, insert an LMA, and perform endotracheal intubation. Paired observations from six facultyevaluators determined inter-rater reliability. Separate survey questionnaires were created for faculty-evaluators and SRNAs; using three four-point Likert-type rating scales to measure feasibility and validity. Results: Descriptive statistics revealed that faculty-evaluators' found the OSATs tools to be feasible based on six out of seven somewhat favorable to strongly favorable responses. The Kendall's Coefficient of Concordance W showed statistical significance for mask ventilation and endotracheal intubation (p-value of .041 and .036). Six faculty-evaluators found the OSATs tools to have face and content validity based on six out of seven somewhat favorable responses. Twenty-three SRNAs found the OSATS tools to have face validity based on somewhat strongly favorable responses. Conclusions: An adapted version of OSATS tool for evaluating SRNAs performing mask ventilation, LMA insertion, and endotracheal intubation demonstrated feasibility, reliability, and validity. The inter-rater values obtained from Kendall's Coefficient of Concordance W in this project correlated with the Cook and Beckman report.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Strategies for evaluating student clinical performance are frequently criticized due to subjectivity. [2] A common rating method for evaluation of clinical skills is the checklist. The scoring is either a pass/fail or yes/no grade.…”

Section: Introductionmentioning

confidence: 99%

Testing an objective structured assessment technical skills tool: A pilot study

Dixon¹,

Burns²

2015

JNEP

View full text Add to dashboard Cite

show abstract

“…Due to a lack of formal rater training in the use of a weighted scoring system, there was a concern that subjectivity could enter into the evaluation process. According to Hayden et al, [5] many of the instruments used to evaluate students and their performance in clinical are thought to be subjective in nature.…”

Section: Discussionmentioning

confidence: 99%

“…However, few instruments, including those used in high-stakes testing like OSCEs, have had rigorous reliability and validity testing prior to utilization. [5] Others report that often in the literature, there is a lack of statistical support for reliability and validity of instruments that is presented. [1] In 2010, Kardong-Edgren et al [6] provided a review of evaluation instruments used in nursing simulations.…”

Section: Review Of Literaturementioning

confidence: 99%

“…[1] To accomplish this, nurse educators are conducting simulation-based evaluations, [2] including those to assess performance. [3][4][5][6][7] Some are labeled as Objective Structured Clinical Examinations/Evaluations (OSCEs) [7][8][9] and others as high-stakes tests or evaluations. [10,11] One of the benefits of patient simulations is that these can be specifically designed for students to demonstrate learning at the application, synthesis, and evaluation levels, as well as provide for a "realistic" yet safe environment for psychomotor skills performance and evaluation of values, attitudes, and beliefs.…”

Section: Introduction 11 Backgroundmentioning

confidence: 99%

See 1 more Smart Citation

Development of a valid and reliable evaluation instrument for undergraduate nursing students during simulation

Stiller

Nelson

Anderson

et al. 2015

JNEP

View full text Add to dashboard Cite

Background: Many nurse educators are using instruments to evaluate undergraduate nursing students' performance during simulation. Rigorous reliability and validity testing for such instruments is often not presented. Evaluation and testing of instruments used is needed to support objective measurement of student performance. Methods: This paper describes the development and testing of three new scenario-specific checklist instruments used for evaluating undergraduate nursing students' performance on an Objective Structured Clinical Examination using simulation. The new instruments were compared with original weighted ones previously used. An informal Delphi method was used to enhance content validity; Cronbach's alpha was utilized for reliability testing. Results: Although neither set of instruments had been used with trained raters, the new set of instruments performed better on reliability analysis. Conclusions: Checklist instruments may be more reliable in the objective measurement of student performance.

show abstract

Addressing the Perioperative Nursing Shortage Via a Perioperative Nursing Preceptorship for Baccalaureate Nursing Students

Manz¹,

Williams

Jessen

et al. 2020

AORN Journal

View full text Add to dashboard Cite

A growing demand for perioperative nurses, insufficient interest in the perioperative nursing specialty, and nurse retirements have resulted in a perioperative nursing workforce shortage. Undergraduate nursing students’ limited exposure to perioperative content, along with facility hiring practices that exclude recently graduated nurses, further contribute to this shortage. To address these concerns, a large health care system in the midwestern United States partnered with a college of nursing to institute a perioperative preceptorship for baccalaureate nursing students during their final semester. After completing the preceptorship, students reported increased understanding of perioperative nursing roles and responsibilities, time management, and patient safety skills; ability to work effectively as a part of an interdisciplinary team; and independence in the perioperative setting. Future study is needed to examine rates of matriculation among program graduates into perioperative nursing positions and the program’s effect on the length of orientation for recently graduated nurses and nurse retention.

show abstract

Reliability and Validity Testing of the Creighton Competency Evaluation Instrument for Use in the NCSBN National Simulation Study

Abstract: The CCEI is useful for evaluating both the simulation and traditional clinical environments.

Cited by 106 publications

References 1 publication

Testing an objective structured assessment technical skills tool: A pilot study

Testing an objective structured assessment technical skills tool: A pilot study

Development of a valid and reliable evaluation instrument for undergraduate nursing students during simulation

Addressing the Perioperative Nursing Shortage Via a Perioperative Nursing Preceptorship for Baccalaureate Nursing Students

Contact Info

Product

Resources

About