Automatic Estimation of Ulcerative Colitis Severity by Learning to Rank With Calibration

Abstract

For automatic disease-severity-level estimation, a large-scale medical image dataset with level annotations is generally necessary. However, attaching absolute-level annotations (such as levels 0, 1, and 3) is very costly and even inaccurate due to the level ambiguity. In this study, we proved experimentally that using a ranking function for level estimation can relax this difficulty. We propose a multi-task learning method for automatically estimating disease-severity levels that combine learning to rank with regression. The ranking function of the proposed method is trainable by relative-level and a small number of absolutelevel annotations. For relative-level annotation, an annotator only needs to specify that one image has a higher disease level than another-this is much easier than absolute-level annotation. The proposed method enables disease-severity classification by calibrating the ranking function based on relative-level annotation through regression. The effectiveness of the method was proved through a large-scale experiment of ulcerative colitis-severity estimation with colonoscopy images.

A systematic review and meta-analysis of artificial intelligence-diagnosed endoscopic remission in ulcerative colitis

“…The remaining 53 articles were read in detail, of which 18 were included. 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 The literature screening process is illustrated in Figure 1 . Details of the included studies are presented in Table 1 .…”

“…Twelve studies were retrospective and six were prospective 28 , 30 , 31 , 34 , 35 , 40 ; fourteen studies were single-center and four were multicenter 30 , 31 , 33 , 40 ; and nine studies were from Europe and America and nine were from Asia. 26 , 28 , 29 , 33 , 35 , 38 , 39 , 42 , 43 All the studies used images obtained from common white-light imaging and used DL algorithms. It should be noted that of the six prospective studies, only Takenaka 2020 28 explicitly stated that the AI model was involved in the actual diagnostic process of the patient, and the other five studies only prospectively collected data for AI testing.…”

“…We will analyze this study separately in the sensitivity analysis. Three studies were considered to have an unknown risk of bias in the patient selection section: two study 29 , 42 did not describe the patient selection process and one study 39 did not describe the quality of the endoscopic images.

Figure 2 Summary of risk of bias and applicability of concerns graph

…”

Deep Bayesian Active-Learning-to-Rank for Endoscopic Image Data

“…We consider this setting reasonable since it used all the original samples, and we could conduct the experiments in a realistic running time. Also, note that this setting is typical for evaluating learning-to-rank [6,15,17].…”

“…However, this ambiguity was minimized through careful observation by multiple experts. In other words, reliable absolute annotation is very costly, and this fact is the motivation of this work 6. This study was approved by the Ethics Committee of the Kyoto Second Red Cross Hospital.…”