Towards Reducing Aleatoric Uncertainty for Medical Imaging Tasks

Sambyal, Abhishek Singh; Krishnan, Narayanan C.; Bathula, Deepti R.

doi:10.1109/isbi52829.2022.9761638

Cited by 9 publications

(3 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The AU does not decrease with increasing training data. It can be reduced by improving the data extraction process to decrease the inherent noise and clarify the features in the data [20], [21]. Hence, the effectiveness of the proposed technique in enhancing image data quality can be verified if the AU values from the MDS images generated utilizing the proposed technique are lower than those from the MDS images generated utilizing the conventional technique.…”

Section: Related Workmentioning

confidence: 99%

Classification of Small Drones Using Low-Uncertainty Micro-Doppler Signature Images and Ultra-Lightweight Convolutional Neural Network

Park,

Park

2024

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Many studies have attempted to classify small drones in response to threats posed by the technical progress of small drones. Recently, small drones have been classified utilizing convolutional neural networks (CNNs) with micro-Doppler signature (MDS) images generated from frequency-modulated continuous-wave (FMCW) radars. This study proposes a comprehensive method for classifying small drones in real-time using high-quality MDS images and an ultra-lightweight CNN. The proposed comprehensive method comprises an MDS image generation technique, which can improve the quality of MDS images generated via FMCW radars, and the ultra-lightweight CNN with high accuracy performance despite its remarkable lightness. Experimental results show that the proposed MDS image generation technique increases the accuracy of CNNs by enhancing the MDS image quality. This is further verified using the results of uncertainty quantification. The proposed ultra-lightweight CNN significantly decreases the computational cost while achieving high accuracy. Finally, we demonstrate that the proposed comprehensive method successfully classifies small drones from far distances with high efficiency and accuracy: the maximum and average accuracies for classification are 100% and 99.21%, respectively, and the numbers of parameters, nodes, and floating-point operations of the proposed ultra-lightweight CNN are approximately 4.88 K, 21.51 K, and 31.52 M, respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Classification of Small Drones Using Low-Uncertainty Micro-Doppler Signature Images and Ultra-Lightweight Convolutional Neural Network

Park,

Park

2024

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

show abstract

“…In the vision community, aleatoric uncertainty is typically considered in the context of medical image processing, where model calibration is necessary to employ agents in high-stakes applications. A popular method of quantifying aleatoric uncertainty in medical imaging is data augmentation [5,62], but authors such as Beluch et al [9] and Reinhold et al [56] use alternate techniques such as ensembling and dropout network layers. Nado et al [47] produce a system for benchmarking such methods, but does not consider using human uncertainty scores to assess models.…”

Section: Assessing Human Uncertainty In Ambiguous Datamentioning

confidence: 99%

Ambiguous Images With Human Judgments for Robust Visual Event Classification

Sanders¹,

Kriz²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Contemporary vision benchmarks predominantly consider tasks on which humans can achieve near-perfect performance. However, humans are frequently presented with visual data that they cannot classify with 100% certainty, and models trained on standard vision benchmarks achieve low performance when evaluated on this data. To address this issue, we introduce a procedure for creating datasets of ambiguous images and use it to produce SQUID-E ("Squidy"), a collection of noisy images extracted from videos. All images are annotated with ground truth values and a test set is annotated with human uncertainty judgments. We use this dataset to characterize human uncertainty in vision tasks and evaluate existing visual event classification models. Experimental results suggest that existing vision models are not sufficiently equipped to provide meaningful outputs for ambiguous images and that datasets of this nature can be used to assess and improve such models through model training and direct evaluation of model calibration. These findings motivate large-scale ambiguous dataset creation and further research focusing on noisy visual data. 1 1 Dataset and code are available at https://katesanders9.github.io/ambiguous-images. 36th Conference on Neural Information Processing Systems (NeurIPS 2022) Track on Datasets and Benchmarks.

show abstract

“…Our definition of ambiguity is also often confused or conflated with uncertainty. The literature mainly distinguishes between aleatoric and epistemic uncertainty [33,163], while acknowledging that it is difficult to distinguish between them in DL [1,83,144,178]. Aleatoric uncertainty is a statistical uncertainty that is inherent in the data and cannot be influenced by the model.…”

Section: Aleatoric Uncertainty and Epistemtic Uncertaintymentioning

confidence: 99%

Addressing the Challenge of Ambiguous Data in Deep Learning

Schmarje

View full text Add to dashboard Cite

In machine learning, the availability of high-quality labeled data is essential for training accurate models. However, humans often disagree among themselves or over time when labeling or annotating image classification data. As a result, they create ambiguous data that poses a significant challenge to deep learning. This research focuses on addressing this issue by proposing an overview that defines the problem, provides the necessary data for research, establishes evaluation metrics and presents methods for solving the problem. The ultimate goal is to develop an effective annotation strategy for image classification based on several thousand different experiments to improve data quality while reducing cost. The main idea of this dissertation is not to rely on one annotation per image, but to consider the average of multiple annotations. However, this would not be feasible in reality without reducing the cost of acquiring such annotations. This dissertation uses proposals as a guide during the annotation process to improve the consistency of the annotations and to reduce the annotation time and thus the associated costs. The use of proposals could introduce a bias into the data. Analysis of this bias and its introduction led to possible methods to minimize or reverse it. All this research is finally unified in an annotation strategy in the form of a flowchart that allows future research to easily understand the necessary steps needed to produce high quality data for their specific use case with as little effort as possible. This strategy is successfully verified on a real biomedical task. In conclusion, this thesis presents a comprehensive investigation of the challenges posed by annotating ambiguous data in deep learning. This research provides an effective strategy to address the fundamental issue of data quality, thereby improving the quality and reliability of models and advancing the field of deep learning as a whole.

show abstract

Towards Reducing Aleatoric Uncertainty for Medical Imaging Tasks

Cited by 9 publications

References 2 publications

Classification of Small Drones Using Low-Uncertainty Micro-Doppler Signature Images and Ultra-Lightweight Convolutional Neural Network

Classification of Small Drones Using Low-Uncertainty Micro-Doppler Signature Images and Ultra-Lightweight Convolutional Neural Network

Ambiguous Images With Human Judgments for Robust Visual Event Classification

Addressing the Challenge of Ambiguous Data in Deep Learning

Contact Info

Product

Resources

About