Multi-Task Classification and Segmentation for Explicable Capsule Endoscopy Diagnostics

Abstract

Capsule endoscopy is a leading diagnostic tool for small bowel lesions which faces certain challenges such as time-consuming interpretation and harsh optical environment inside the small intestine. Specialists unavoidably waste lots of time on searching for a high clearness degree image for accurate diagnostics. However, current clearness degree classification methods are based on either traditional attributes or an unexplainable deep neural network. In this paper, we propose a multi-task framework, called the multi-task classification and segmentation network (MTCSN), to achieve joint learning of clearness degree (CD) and tissue semantic segmentation (TSS) for the first time. In the MTCSN, the CD helps to generate better refined TSS, while TSS provides an explicable semantic map to better classify the CD. In addition, we present a new benchmark, named the Capsule-Endoscopy Crohn’s Disease dataset, which introduces the challenges faced in the real world including motion blur, excreta occlusion, reflection, and various complex alimentary scenes that are widely acknowledged in endoscopy examination. Extensive experiments and ablation studies report the significant performance gains of the MTCSN over state-of-the-art methods.

A multi-task convolutional neural network for classification and segmentation of chronic venous disorders

“…The proposed VENet was compared against state-of-the-art multi-task learning methods in this experiment. Specifically, four different methodologies were tested, namely MTCSN 35 , DSI-Net 36 , Le et. al.…”

“…Second, CVD presents a wide range of sizes and shapes, ranging from small vessels with 1 mm of caliber to varicose veins often protruding from the skin to skin ulcers with well-defined features. Interestingly, and although not validated in CVD, for high variable segmentation/classification situations, multi-task learning techniques have recently demonstrated their added value to empower network training [32][33][34][35][36] . These methodologies combine information from different tasks, boosting the generalization ability of the network, and showing improvement in the overall method's performance.…”

Improved vision-based diagnosis of multi-plant disease using an ensemble of deep learning methods

“…With the fast improvement of computer vision-based diagnosis approaches, deep learning techniques are widely utilized for recognizing plant disease efficiently [1]. In order to enhance the accuracy and diagnosis speed of plant disease recognition techniques, researchers designed and evaluated numerous convolutional neural networks (CNN), which provided remarkable performance in semantic understanding and visual recognition [2]. In various computer vision-based diagnosis applications, CNN is widely applied for its efficient learning capacity, which is able to learn high-level robust feature representations directly from raw images [3].…”

“…Kong et al [2] addressed a multi-task learning method for diagnosing Crohn's disease, and their proposed architecture, namely, multi-task classification and segmentation network (MTCSN), obtained 89.23% accuracy using ResNet50, where MTCSN acquired 84.75% and 88.30% accuracy using ResNet101and DenseNet121, respectively. For image sentiment analysis, Moung et al [18] addressed an ensemble-based method of facial expression recognition, where three deep learning models were combined with an averaging technique.…”