Validity of In-Training Evaluation Report (ITER) in Saudi Pediatric Residency Training. Single center experience

Abstract

Objective: The study aimed to examine validity evidence of in-training evaluation report (ITER) at King Abdulaziz University Hospital (KAUH)'s pediatric residency program. Predictive validity of ITERs, to assess learner acquisitions of required competencies were studied. We used ITER scores to identify trainees with unprofessional conduct. Methods:One-year data for 35 residents in pediatric residency training program at KAUH were reviewed. Data were extracted from 360 ITERs completed by faculty members and specialists. ITERs' reliability was evaluated using Cronbach's alpha. ITERs scores were correlated with final Objective Structured Clinical Examinations (OSCE) and written exams scores. Professionalism composite scores from ITERs were compared with scores of trainees reported for unprofessional attitude.Results: Specialists had high variability in scoring competencies with average standard deviations (SD) of 0.85 compared with 0.15 by faculty. Reliability of ITER scores using Cronbach's alpha was 0.895. Good correlation was observed between competencies ranging between 0.86 to 0.29. There was fair correlation between ITER composite scores and those of final OSCE and written exams. There was no correlation between professionalism scores and residents reported for unprofessional conduct.Conclusion ITER scores had good validity and reliability despite variability observed between faculty and specialist when scoring competencies. ITERs failed to detect trainees with unprofessional conduct.

Pediatric residents’ perceptions of the impact of the 24-hour on-call system on their well-being and education and patient safety

“…Since the commission's inception, the number of accredited training centers and trainees has grown consistently. 14 The Saudi Commission for Health Specialties outlined some guidelines for duty hour regulations for the pediatrics program where it holds the long-standing tradition; physicians in training complete long hours of on-call work, including working shifts longer than 24 hours. The Pediatric Residency Training Program Manual states that the mean number of calls for year one and 2 residents should be limited to 7 every month whereas those for year 3 and 4 residents should be no more than 6 every month.…”