Introduction of direct observation of procedural skills (DOPS) as a formative assessment tool during postgraduate training in anaesthesiology

Abstract

Background and Aims: Formative assessment of procedural skills of Anaesthesiology postgraduate (PG) students is not conducted conventionally. Direct observation of procedural skills (DOPS) helps to identify gaps in performance and provides structured feedback. The present study was taken to explore perceptions of PG students and faculty about DOPS. Methods: This mixed design interventional study was conducted on 12 PGs and 10 faculty members in Department of Anaesthesiology. After conducting DOPS, a pre-validated questionnaire was given to them about perceptions using 3-point Likert's scale along with open ended questions. Statistical analysis was done using descriptive statistics of perception to calculate percentages and themes were identified for qualitative data. Results: Responses of students were positive about skill improvement (83%), time provided (75%), feedback (100%), interaction (83%), motivation (83%), satisfaction (83%), effectiveness (83%) and opportunity creation (92%). Faculty responded positively regarding change in attitude (100%), effectiveness (100%), scope (90%), feasible application (90%), ease (90%), opportunity creation (80%), gap identification (100%), satisfaction (80%). However, 60% felt training was required, 50% thought more time and commitment was required. Themes identified were DOPS is comprehensive, interactive, student-friendly, good teaching-learning tool, identifies gaps, provides focus for learning, provides systematic constructive feedback, improves skills, prepares for future practice, requires planning, may not reflect competence, has assessor variability and can be included in PG curriculum. Conclusion: DOPS was perceived as an effective assessment and teaching-learning tool by PG students as well as faculty.

An action research to introduce direct observation of procedural skills (DOPS) as clinical skill performance assessment tool during medical internship in the department of surgery, Debre Tabor University, Debre Tabor, Ethiopia, 2023

“…This is consistent with the other studies that showed similar feedbacks (Zainab Shafiq et al, 2019). As most of other studies also, DOPS satisfaction and practicality in this study were reported as favorable (Bangal, 2018;Jui and Shilpa, 2021;McLeod et al, 2011;Wilkinson et al, 2008).…”

“…This study also found an association between progressive DOPS practices and improved trainees' performance as many comparable finding showed similar results (Lörwald et al, 2018;Jui and Shilpa, 2021). Furthermore, major challenges to implement this method of assessment met by participants of this study included lack of awareness and work overload.…”

“…DOPS are not designed to test the person but rather provide the opportunity for that to ensure that a particular skill is performed correctly according to agreed guidelines using an agreed checklist in the workplace setting (Duffy et al, 2004). In DOPS, the trainee is evaluated regarding his or her demonstrated understanding of indications, relevant anatomy, technique of procedure, obtaining informed consent, demonstrating appropriate preparation pre-procedure, technical ability, aseptic technique, seeking help where appropriate, post-procedure management, communication skills, consideration of patient/professionalism, and overall ability to perform the procedure involving real patient encounter (Lörwald et al, 2018;Jui and Shilpa, 2021). Here is a concern that undergraduate medical students are seldom observed, assessed and given feedback during their workplace clinical attachments and their procedural skill performance.…”

“…Workplace-based assessment involves ongoing formative assessment of knowledge, clinical decision-making and procedural skills (Wragg et al, 2003). DOPS helps to identify gaps in performance and provides structured feedback (Jui and Shilpa, 2021).…”

Application of direct observation of operational skills in nursing skill evaluation of pressure injury: A randomized clinical trial

“… 14 It has been successfully applied by Siau K and others to assess skills related to gastroscopy and colonoscopy. 15 , 16 Lagoo JY and colleagues introduced DOPS as a formative assessment tool during anaesthesia residency training, 17 and Ali L and his team validated its effectiveness in urological surgery resident training. 18 While DOPS is extensively used in nursing abroad, 19 , 20 its application in nursing skills is still in the early exploration stage in China, with limited use in specialized nursing skill evaluation.…”