MIXCAPS: A capsule network-based mixture of experts for lung nodule malignancy prediction

Afshar, Parnian; Naderkhani, Farnoosh; Oikonomou, Anastasia; Rafiee, Moezedin Javad; Mohammadi, Arash; Plataniotis, Konstantinos N.

doi:10.1016/j.patcog.2021.107942

Cited by 44 publications

(13 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regarding the technical approach to provide transparency, the incorporation of medical experts motivated designers to incorporate prior knowledge directly into the model structure and/or inference for medical imaging (73%/64% articles with/without the incorporation of end users do not need a second model to generate transparency). enabled the generation of pixel-attribution methods 54 to visualize pixel-level importance for a specific class of interest [55][56][57][58][59][60][61][62][63][64][65][66][67][68][69] . In segmentation tasks, where clinically relevant abnormalities and organs are usually of small sizes, features from different resolution levels were aggregated to compute attention and generate more accurate outcomes, as demonstrated in multiple applications, e.g., multi-class segmentation in fetal Magnetic Resonance Imagings (MRIs) 58 and multiple sclerosis segmentation in MRIs 61 .…”

Section: In: Incorporationmentioning

confidence: 99%

“…Image visualization-based techniques to achieve transparency were most commonly considered in image classification problems. Common ways of retrieving relevance information were: Visual relevancy through attention [55][56][57][58][59][60][61][62][63][64][66][67][68] ; region occlusion by blank areas 109,111 or healthy-looking regions 112 ; and other techniques such as supervision of activated image regions by clinically relevant areas 88,89,92,94,95,97,98 , and image similarity 96 . Feature-based computer vision transparency priors focused on the impact of feature evolution or perturbation on the decoded output.…”

Section: Pr: Priorsmentioning

confidence: 99%

See 1 more Smart Citation

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review

et al. 2022

View full text Add to dashboard Cite

Transparency in Machine Learning (ML), often also referred to as interpretability or explainability, attempts to reveal the working mechanisms of complex models. From a human-centered design perspective, transparency is not a property of the ML model but an affordance, i.e., a relationship between algorithm and users. Thus, prototyping and user evaluations are critical to attaining solutions that afford transparency. Following human-centered design principles in highly specialized and high stakes domains, such as medical image analysis, is challenging due to the limited access to end users and the knowledge imbalance between those users and ML designers. To investigate the state of transparent ML in medical image analysis, we conducted a systematic review of the literature from 2012 to 2021 in PubMed, EMBASE, and Compendex databases. We identified 2508 records and 68 articles met the inclusion criteria. Current techniques in transparent ML are dominated by computational feasibility and barely consider end users, e.g. clinical stakeholders. Despite the different roles and knowledge of ML developers and end users, no study reported formative user research to inform the design and development of transparent ML models. Only a few studies validated transparency claims through empirical user evaluations. These shortcomings put contemporary research on transparent ML at risk of being incomprehensible to users, and thus, clinically irrelevant. To alleviate these shortcomings in forthcoming research, we introduce the INTRPRT guideline, a design directive for transparent ML systems in medical image analysis. The INTRPRT guideline suggests human-centered design principles, recommending formative user research as the first step to understand user needs and domain requirements. Following these guidelines increases the likelihood that the algorithms afford transparency and enable stakeholders to capitalize on the benefits of transparent ML.

show abstract

Section: In: Incorporationmentioning

confidence: 99%

Section: Pr: Priorsmentioning

confidence: 99%

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Alternatively, achieving transparency in an ML model by revealing its working mechanisms is widely hypothesized to invoke user trust in ML systems [97,16]. There have been approaches to provide transparency into decision making processes by incorporating prior knowledge directly into the model structure and/or inference process in hopes of invoking interpretability [3,18] or providing post-hoc explanations for black box models [24,54]. However, as we will highlight in detail through a systematic review, current approaches that aim at incorporating transparency to ML systems rely on developers' intuition on what may be transparent, rather than considering whether these mechanisms affect users' experience with the system and their ability to interpret ML model's outputs.…”

Section: Machine Learning For Healthcarementioning

confidence: 99%

“…The complex nature of both 3D imaging in radiology and pathological images makes image analysis tasks more time consuming than 2D image analysis that is more prevalent in other specialities, such as dermatology, which motivates transparency as an alternative to complete human image analysis to save time while retaining trustworthiness. In detail, classification problems in 3D radiological images and pathological images included abnormality detection in computed tomography (CT) scans [3,5,61,47,89,107,111,112], MRIs [34,11,38,51,59,87,85,50,95,77,78,98,100,104], pathology images [1,24,26,27,30,34,37,40,82,84,50,74,76,108,5] and positron emission tomography (PET) images [68]. Mammography dominated the 2D radiology image applications [60,88,44,45,86,96,99,…”

Section: T: Taskmentioning

confidence: 99%

INTRPRT: A Systematic Review of and Guidelines for Designing and Validating Transparent AI in Medical Image Analysis

Chen¹,

Gómez²,

Huang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Transparency in Machine Learning (ML), including interpretable or explainable ML, attempts to reveal the working mechanisms of complex models, including deep neural networks. Transparent ML promises to advance human factors engineering goals of human-centered AI, such as increasing trust or reducing automation bias, in the target users. However, a core requirement in achieving these aims is that the method is indeed transparent to the users. From a human-centered design perspective, transparency is not a property of the ML model but an affordance, i. e., a relationship between algorithm and user; as a result, iterative prototyping and evaluation with users is critical to attaining adequate solutions that afford transparency. However, following human-centered design principles in highly specialized and high stakes domains, such as healthcare and medical image analysis, is challenging due to the limited availability of and access to end users, such as domain experts, providers, or patients. This dilemma is further exacerbated by the high knowledge imbalance between ML designers and those end users, which implies an even greater need for iterated development.To investigate the state of transparent ML in medical image analysis, we conducted and now present a systematic review of the literature. Our review reveals multiple severe shortcomings in the design and validation of transparent ML for medical image analysis applications. We find that most studies to date approach transparency as a property of the model itself, similar to task performance, without considering end users during neither development nor evaluation. Despite the considerable difference between the roles and knowledge of ML developers and clinical stakeholders, no study reported formative user research to inform the design and development of transparent ML models; moreover, only a few studies validated transparency claims through empirical user evaluations. The oversight of considering ML transparency as a relationship with end users, the lack of user research, and the sporadic validation of transparency claims put contemporary research on transparent ML for medical image analysis at risk of being incomprehensible to users, and thus, clinically irrelevant. To alleviate these shortcomings in forthcoming research while acknowledging the challenges of human-centered design in healthcare, we introduce the INTRPRT guideline, a systematic design directive for transparent ML systems in medical image analysis. To bridge the disconnect between ML system designers and end users and avoid misspending costly development efforts on models that are finally validated as non-transparent to end users, the INTRPRT guideline suggests formative user research as the first step of transparent model design to understand user needs and domain requirements. Following this process produces evidence to support * Joint first authors.Preprint. Under review.

show abstract

“…The input of Stage 1 is the normalized lung area as a 2D image and the output is the label indicating whether the input image demonstrates infection or not. The classification model used in this stage is based on the Capsule Networks (CapsNets) 8 , which have shown a superior discriminative capability compared to 9/14 their CNN-based counterparts, especially when they are trained over small datasets [39][40][41][42] . Each capsule layer consists of multiple capsules, which are groups of neurons represented by a vector.…”

Section: Capsule Network-based Frameworkmentioning

confidence: 99%

Robust Automated Framework for COVID-19 Disease Identification from a Multicenter Dataset of Chest CT Scans

Khademi¹,

Heidarian²,

Afshar³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The objective of this study is to develop a robust deep learning-based framework to distinguish COVID-19, Community-Acquired Pneumonia (CAP), and Normal cases based on chest CT scans, which are acquired in different imaging centers using various scanners, scanning protocols, and radiation doses. We showed that while our proposed model is trained on a relatively small dataset acquired from only one imaging center using a specific scanning protocol, the model performs well on heterogeneous test sets obtained by multiple scanners using different technical parameters. We also showed that the model can be updated via an unsupervised approach to cope with the data shift between the train and test sets and enhance the robustness of the model upon receiving a new external dataset from a different center. More specifically, we extracted the subset of the test images for which the model generated a confident prediction and used the extracted subset along with the training set to re-train and update the benchmark model (the model trained on the initial train set). Finally, we adopted an ensemble architecture to aggregate the predictions from multiple versions of the model. For initial training and development purposes, an in-house dataset of 171 COVID-19, 60 CAP, and 76 Normal cases was used, which contained volumetric CT scans acquired from one imaging center using a constant standard radiation dose scanning protocol. To evaluate the model, we collected four different test sets retrospectively to investigate the effects of the shifts in the data characteristics on the model's performance. Among the test cases, there were CT scans with similar characteristics as the train set as well as noisy low-dose and ultra-low dose CT scans. In addition, some test CT scans were obtained from patients with a history of cardiovascular diseases or surgeries. The entire test dataset used in this study contained 51 COVID-19, 28 CAP, and 51 Normal cases. Experimental results indicate that our proposed framework performs well on all test sets achieving the a total accuracy of 96.15% (95%CI : [91.25 − 98.74]), COVID-19 sensitivity of 96.08% (95%CI : [86.54 − 99.5]), CAP sensitivity of 92.86% (95%CI : [76.50 − 99.19]), Normal sensitivity of 98.04% (95%CI : [89.55 − 99.95]) while the confidence intervals are obtained using the significance level of 0.05. The obtained AUC values (One class vs Others) are 0.993 (95%CI : [0.977 − 1]), 0.989 (95%CI : [0.962 − 1]), and 0.990 (95%CI : [0.971 − 1]) for COVID-19, CAP, and Normal classes, respectively. The experimental results also demonstrate the capability of the proposed unsupervised enhancement approach to improve the performance and robustness of the model when being evaluated on varied external test sets.

show abstract

MIXCAPS: A capsule network-based mixture of experts for lung nodule malignancy prediction

Cited by 44 publications

References 20 publications

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review

Explainable medical imaging AI needs human-centered design: guidelines and evidence from a systematic review

INTRPRT: A Systematic Review of and Guidelines for Designing and Validating Transparent AI in Medical Image Analysis

Robust Automated Framework for COVID-19 Disease Identification from a Multicenter Dataset of Chest CT Scans

Contact Info

Product

Resources

About