Predicting Pouch of Douglas Obliteration Using Ultrasound and Laparoscopic Video Sets: An Interobserver and Diagnostic Accuracy Study

Abstract

Objectives-To investigate the diagnostic accuracy and interobserver agreement among sonologists when assessing offline ultrasound (US) video sets of the "sliding sign" and among gynecologic surgeons when assessing corresponding laparoscopic video sets to predict pouch of Douglas (POD) obliteration and to compare the performance of the groups.Methods-A diagnostic and reproducibility study was conducted, including 15 observers in 4 groups: (1) senior sonologists, (2) junior sonologists, (3) general gynecologists, and (4) advanced laparoscopists. The sonologists viewed 25 offline preoperative US video sets of the sliding sign, and the surgeons viewed the corresponding intraoperative laparoscopic videos of the same patients. Each observer was asked to classify POD obliteration in the video sets and was compared to the reference standard POD state determined at real-time laparoscopy by a single investigator (G.C.). The interobserver correlation and diagnostic accuracy were evaluated among the 15 observers and 4 groups. The Cohen κ coefficient and Fleiss κ coefficient were used for the analysis.Results-The overall accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for senior sonologists were 93.3%, 100%, 89.6%, 84.4%, and 100%, respectively; for junior sonologists, 70.0%, 88.9%, 59.4%, 55.2%, and 90.5%; for general gynecologists, 75.2%, 88.1%, 78.1%, 69.8%, and 91.9%; and for advanced laparoscopists, 82.4%, 91.9%, 90.8%, 82.9%, and 95.8%. The overall agreement between senior sonologists was almost perfect (Fleiss κ = 0.876); for junior sonologists and general gynecologists, it was moderate (Fleiss κ = 0.589 and 0.528); and for advanced laparoscopists, it was substantial (Fleiss κ = 0.652).Conclusions-Interobserver agreement was superior among senior sonologists. Prediction of POD obliteration using offline US videos by senior sonologists is comparable to offline assessments of laparoscopic videos by advanced laparoscopists for prediction of POD obliteration.

Prevalence of negative sliding sign representing pouch of Douglas obliteration during pelvic transvaginal ultrasound for any indication

“…The use of the indeterminate classification category was meant to safeguard against this limitation. Importantly, we have demonstrated previously that senior sonologists (EFSUMB Level‐3) have very high accuracy (93.3%) and reproducibility (almost perfect agreement, Fleiss κ, 0.876) in classifying the POD state using recorded sliding sign videoclip sets 23 , which should overcome the lack of surgical data deficiency. It could also be argued that utilizing as reference standard the interpretation of our expert sonologist limits the generalizability of our findings, as less experienced observers may have lower accuracy.…”

One‐Size‐Fits‐All Approach Does Not Work for Gynecology Trainees Learning Endometriosis Ultrasound Skills

“…12 The expert-guided TVUS examination recommended by the International Deep Endometriosis Analysis (IDEA) group 11 includes the sliding sign technique, which has been demonstrated as one of the most effective means of assessing POD obliteration. 13,14 Most US centers will only perform the basic TVUS examination, comprising the uterus, ovaries, and POD, for fluid or a mass, 15 forgoing the specialized techniques to ascertain POD obliteration or DE. Beyond the medical benefits for patients, it has also been demonstrated to be cost-effective to introduce expert-guided TVUS into the public health care system.…”

“…Although diagnostic laparoscopy with a histologic analysis currently defines the reference standard for diagnosis of endometriosis, 7,8 transvaginal ultrasound (TVUS) is recommended as the first‐line investigation of choice because of its availability, low cost, and patient acceptability 9–11 and is rapidly challenging the necessity of laparoscopy as a diagnostic tool 12 . The expert‐guided TVUS examination recommended by the International Deep Endometriosis Analysis (IDEA) group 11 includes the sliding sign technique, which has been demonstrated as one of the most effective means of assessing POD obliteration 13,14 …”

Deep learning to diagnose pouch of Douglas obliteration with ultrasound sliding sign

“…Another limitation in the study is that we did not have surgical data confirming the state of the POD. However, the diagnostic accuracy of sonologist-interpreted sliding sign is high ( Nisenblat et al 2016 ) and there is evidence of almost perfect interobserver agreement of expert sonologists interpreting offline videos of the sliding sign ( Chiu et al 2019 ). A study involving all patients that undergo surgery would be advantageous, but it will be limited by the high prevalence of pathology that necessitates the surgery in the first place.…”

“…The sliding sign is an accurate dynamic transvaginal ultrasound (TVS) test that is used to evaluate the POD ( Hudelist et al 2013 , Reid et al 2013 ). It can be interpreted by an ultrasound operator at the time of point-of-care scanning or by a radiologist/sonologist observing the recorded videos ( Chiu et al 2019 ). The dynamic nature of TVS mandates that in order to perform the sliding sign correctly, one must have adequate knowledge of normal (producing a positive sliding sign) and abnormal (producing a negative sliding sign) female pelvic anatomy.…”