Accuracy and reliability of qualitative echocardiography assessment of right ventricular size and function in neonates

Abstract

Background Subjective assessment of right ventricular (RV) function by neonatal echocardiography lacks validation. Incorrect diagnostic assignment in patients with suspected pulmonary hypertension (PH) may lead to unnecessary treatment or missed treatment opportunities. Methods Six evaluators (experts [n = 3], novice [n = 3]) were asked to independently rate RV characteristics (global function, dilation, and septal flattening) based on standardized echocardiography images. We randomly selected 60 infants, ≥35 weeks gestation at birth, of whom 30 were clinically unwell with acute pulmonary hypertension (aPH) and 30 were healthy controls. aPH was defined by echocardiography presence of right‐left shunting across transitional shunts or elevated right ventricular systolic pressure as estimated by the magnitude of the regurgitant jet across the tricuspid valve with impaired oxygenation. Inter‐rater comparative evaluation within groups and between groups was performed using Kappa statistics. Results Global agreement between evaluators for subjective assessment of RV function (0.3 [0.03], P < 0.001), size (0.14 [0.02], P < 0.001), and septal flattening (0.2 [0.02], P < 0.001) was uniformly poor. Agreement in RV function assessment was marginally better for both expert (0.32 [0.08], P < 0.001 vs 0.13 [0.081], and P < 0.001) and novice (0.4 [0.08], P < 0.001 vs 0.06 [0.07], and P < 0.001) evaluators. Overall, the diagnosis of aPH vs control was misclassified in 18% of cases. Conclusion This study demonstrated significant variability in qualitative assessment of RV size and function by trained evaluators, regardless of level of expertise attained. The reliability of objective measures of RV hemodynamics requires prospective evaluation.

Methodological considerations in accuracy and reliability of qualitative echocardiography assessment of right ventricular size and function in neonates

“…For this aim, the inter-rater reliability was assessed with Cohen's k coefficient. 1 Based on the authors' results, the global agreement between evaluators for subjective assessment of RV function, size, and septal flattening was 0.03, 0.02, and 0.14, respectively. Also, agreement in RV function assessment was marginally better for both expert (0.32 vs 0.13) and novice (0.4 vs 0.06) evaluators.…”

“…We were interested to read an article that was recently published by Smith A and colleagues in the 2019 issue of Echocardiography. 1 The purpose of the authors was to assess the validation of right ventricular (RV) performance evaluation and the reliability of qualitative assessment of RV size, RV function, and septal wall positioning in a representative cohort of term or near-term neonates with pulmonary hypertension (PH) and RV dysfunction versus healthy controls (HC). 1 Six evaluators (experts [n = 3], novice [n = 3]) were asked to independently rate RV characteristics (global function, dilation, and septal flattening) based on standardized echocardiography images.…”

“…Also, agreement in RV function assessment was marginally better for both expert (0.32 vs 0.13) and novice (0.4 vs 0.06) evaluators. 1 We would like to explain some methodological issues about this study. First of all, it is important to know that the concept and definition of reliability (precision, repeatability, and reproducibility) and validity (accuracy) should be correctly considered in reliability researches.…”

Reply to: Methodological considerations in accuracy and reliability of qualitative echocardiography assessment of right ventricular size and function in neonates

“…We appreciated the letter by Naderi and Sabour regarding the use of Cohen's kappa statistic in reference to our recent article “Accuracy and reliability of qualitative echocardiography assessment of right ventricular size and function in neonates.” The authors have made the same arguments about the use of this statistic in several recently published letters about research published in a variety of medical fields …”

“…and reliability of qualitative echocardiography assessment of right ventricular size and function in neonates." 1 The authors have made the same arguments about the use of this statistic in several recently published letters about research published in a variety of medical fields. [2][3][4][5][6] We should clarify that we made a mistake in the reporting of our statistical methods.…”