Inter‐Observer Agreement Among Medical Professionals in Critical Care of Neonates and Children

Abstract

Inter-observer agreement is essential to medical staff members and has a major effect on communication. The goal of the study was to examine the way medical professionals evaluate the potential severity of Almost Adverse Events (AAEs) that were observed in two intensive care units (ICUs). One hundred and fourteen AAEs were observed and recorded in both units by engineering students. Each AAE was rated independently by five senior medical staff members from each ICU, chosen by the unit manager, on a three-point severity level scale. Statistical analysis (K statistic and Cohen's Kappa) yielded relatively low levels of agreement among raters in both ICUs (< 0.3), but significantly greater agreement was found among nurses than among physicians in both ICUs. Low levels of agreement are attributed to the nature of work and characteristics of each ICU. Recommendations for improving agreements including forming shared mental models are specified.