Automatic estimation of ulcerative colitis severity from endoscopy videos using ordinal multi-instance learning

Schwab, Evan; Cula, Gabriela Oana; Standish, Kristopher; Yip, Stephen; Stojmirović, Aleksandar; Ghanem, Louis; Chehoud, Christel

doi:10.1080/21681163.2021.1997644

Cited by 12 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ozawa et al [15] used GoogLeNet [26] to classify images into three EMS levels (Mayo 0, Mayo 1, and Mayo 2-3) due to lack of severe cases. Among the studies in the literature, only Schwab et al [22] used an ordinal classification approach. Their models output a continuous value between 0 and 3, and classes are assigned according to the thresholds; However, class thresholds are determined using a grid search on the test dataset, which limits the generalizability of the proposed method.…”

Section: Related Workmentioning

confidence: 99%

Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation

Polat¹,

Ergenç²,

Kani³

et al. 2022

Preprint

View full text Add to dashboard Cite

Endoscopic Mayo score and Ulcerative Colitis Endoscopic Index of Severity are commonly used scoring systems for the assessment of endoscopic severity of ulcerative colitis. They are based on assigning a score in relation to the disease activity, which creates a rank among the levels, making it an ordinal regression problem. On the other hand, most studies use categorical cross-entropy loss function, which is not optimal for the ordinal regression problem, to train the deep learning models. In this study, we propose a novel loss function called class distance weighted cross-entropy (CDW-CE) that respects the order of the classes and takes the distance of the classes into account in calculation of cost. Experimental evaluations show that CDW-CE outperforms the conventional categorical cross-entropy and CORN framework, which is designed for the ordinal regression problems. In addition, CDW-CE does not require any modifications at the output layer and is compatible with the class activation map visualization techniques.

show abstract

Section: Related Workmentioning

confidence: 99%

Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation

Polat¹,

Ergenç²,

Kani³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Accurate evaluation of treatment effects is important because the type and dosage of treatment medicines are adjusted in accordance with the condition of the patient with UC. Recently, Schwab et al [10] proposed an automatic UC severity estimation method for treatment effectiveness evaluation. Their method assumes a weakly supervised learning scenario because full annotation of all captured images is too costly.…”

Section: Related Workmentioning

confidence: 99%

Deep Bayesian Active-Learning-to-Rank for Endoscopic Image Data

Kadota¹,

Hayashi²,

Bise³

et al. 2022

Preprint

View full text Add to dashboard Cite

Automatic image-based disease severity estimation generally uses discrete (i.e., quantized) severity labels. Annotating discrete labels is often difficult due to the images with ambiguous severity. An easier alternative is to use relative annotation, which compares the severity level between image pairs. By using a learning-to-rank framework with relative annotation, we can train a neural network that estimates rank scores that are relative to severity levels. However, the relative annotation for all possible pairs is prohibitive, and therefore, appropriate sample pair selection is mandatory. This paper proposes a deep Bayesian activelearning-to-rank, which trains a Bayesian convolutional neural network while automatically selecting appropriate pairs for relative annotation. We confirmed the efficiency of the proposed method through experiments on endoscopic images of ulcerative colitis. In addition, we confirmed that our method is useful even with the severe class imbalance because of its ability to select samples from minor classes automatically.

show abstract

“…In this paper, we focus on the ordinal multi-instance-learning problem. The most similar work to ours is the ordinal multi-instance-learning approach proposed by Evan et al [ 12 ]. One difference is that we directly model the bag as multi-class and select the instance with minimum loss from the bags as a positive instance of the bags to update the model parameters.…”

Section: Related Workmentioning

confidence: 99%

“…Here, we consider another typical scenario of MIL-ordinal multiple instance learning [ 12 ] (OMIL): the classes of bags are not only ordered, but the instances in the bag are also in a certain rank order. Moreover, the highest level of instance label cannot exceed the label of the bag.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Uncertainty Ordinal Multi-Instance Learning for Breast Cancer Diagnosis

Xiaoli

Guo

et al. 2022

Healthcare

View full text Add to dashboard Cite

Ordinal multi-instance learning (OMIL) deals with the weak supervision scenario wherein instances in each training bag are not only multi-class but also have rank order relationships between classes, such as breast cancer, which has become one of the most frequent diseases in women. Most of the existing work has generally been to classify the region of interest (mass or microcalcification) on the mammogram as either benign or malignant, while ignoring the normal mammogram classification. Early screening for breast disease is particularly important for further diagnosis. Since early benign lesion areas on a mammogram are very similar to normal tissue, three classifications of mammograms for the improved screening of early benign lesions are necessary. In OMIL, an expert will only label the set of instances (bag), instead of labeling every instance. When labeling efforts are focused on the class of bags, ordinal classes of the instance inside the bag are not labeled. However, recent work on ordinal multi-instance has used the traditional support vector machine to solve the multi-classification problem without utilizing the ordinal information regarding the instances in the bag. In this paper, we propose a method that explicitly models the ordinal class information for bags and instances in bags. Specifically, we specify a key instance from the bag as a positive instance of bags, and design ordinal minimum uncertainty loss to iteratively optimize the selected key instances from the bags. The extensive experimental results clearly prove the effectiveness of the proposed ordinal instance-learning approach, which achieves 52.021% accuracy, 61.471% sensitivity, 47.206% specificity, 57.895% precision, and an 59.629% F1 score on a DDSM dataset.

show abstract

Automatic estimation of ulcerative colitis severity from endoscopy videos using ordinal multi-instance learning

Cited by 12 publications

References 13 publications

Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation

Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation

Deep Bayesian Active-Learning-to-Rank for Endoscopic Image Data

Uncertainty Ordinal Multi-Instance Learning for Breast Cancer Diagnosis

Contact Info

Product

Resources

About