Generalization Evaluation of Machine Learning Numerical Observers for Image Quality Assessment

Kalayeh, Mahdi M.; Marin, Thibault; Brankov, Jovan G.

doi:10.1109/tns.2013.2257183

Cited by 36 publications

(31 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…10,11 Therefore, anthropomorphic model observers (MO) that perform as humans in specific clinical tasks were introduced and are now widely used in the CT research community for the development and optimization of imaging devices and reconstruction algorithms. 12,13 In recent years, machine learning approaches like support vector machines (SVM), [14][15][16][17] relevance vector machines (RVM) [18][19][20] and neural networks, 17 were introduced to build anthropomorphic model observers for task-based image quality assessment of single photon emission computed tomography (SPECT). The authors observed better agreements with human observers of these methods compared to the widely used channelized Hotelling observer (CHO).…”

Section: Introductionmentioning

confidence: 99%

CNN as model observer in a liver lesion detection task for x‐ray computed tomography: A phantom study

et al. 2018

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

CNN as model observer in a liver lesion detection task for x‐ray computed tomography: A phantom study

et al. 2018

View full text Add to dashboard Cite

show abstract

“…RSFM is potentially more capable than JCFO since in RSFM the kernel width parameter is not used for feature selection and so can be employed for improving the performance. Furthermore, multikernel methods [15] are shown to improve the classification performance for example in RVM. RSFM is capable of employing multikernel methods and we are currently extending the proposed RSFM to a multikernel one.…”

Section: A Discussionmentioning

confidence: 99%

“…The numerator of (17) can be calculated by using (12), (14), and (15). The denominator of (17) is the integral of the numerator with respect to variables w and λ.…”

Section: B Inferencementioning

confidence: 99%

“…The corresponding kernel functions of these weights can then be pruned from (1) and sparsity is realized. Success of RVM in regression and classification has been proven in many applications; for example, in hand motion recognition [3], pose estimation and tracking [4], scaling text classification [5], 3-D human pose recovery from silhouettes [6], detection of clustered microclassifications for mammography [7], classification of gene expression data [8], [9], detection of activations for neuroimaging [10], real-time tracking [11], object detection in scenes [12], and mimicking human visual understanding in the cardiac defect detection task [13], [14], [15].…”

mentioning

confidence: 99%

See 1 more Smart Citation

The Relevance Sample-Feature Machine: A Sparse Bayesian Learning Approach to Joint Feature-Sample Selection

Mohsenzadeh

Sheikhzadeh

Azimi

et al. 2013

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Abstract-This paper introduces a novel sparse Bayesian machine-learning algorithm for embedded feature selection in classification tasks. Our proposed algorithm, called the relevance sample feature machine (RSFM), is able to simultaneously choose the relevance samples and also the relevance features for regression or classification problems. We propose a separable model in feature and sample domains. Adopting a Bayesian approach and using Gaussian priors, the learned model by RSFM is sparse in both sample and feature domains. The proposed algorithm is an extension of the standard RVM algorithm, which only opts for sparsity in the sample domain. Experimental comparisons on synthetic as well as benchmark data sets show that RSFM is successful in both feature selection (eliminating the irrelevant features) and accurate classification. The main advantages of our proposed algorithm are: less system complexity, better generalization and avoiding overfitting, and less computational cost during the testing stage.Index Terms-Joint feature selection and classifier design, relevance features, relevance sample feature machine (RSFM), relevance samples, sparse Bayesian learning.

show abstract

“…(2) It allows for partial area under ROC curve analysis, 13 which may have better clinical relevance for a specific level of sensitivity and/or specificity. (3) In our current research on model observer development using a machine learning methodology, 6,[14][15][16] we observed that models perform better if trained on a human confidence rating.…”

Section: Introductionmentioning

confidence: 89%

Full receiver operating characteristic curve estimation using two alternative forced choice studies

Massanes

Brankov

2016

J. Med. Imag

View full text Add to dashboard Cite

Abstract. Task-based medical image quality is typically measured by the degree to which a human observer can perform a diagnostic task in a psychophysical human observer study. During a typical study, an observer is asked to provide a numerical score quantifying his confidence as to whether an image contains a diagnostic marker or not. Such scores are then used to measure the observers' diagnostic accuracy, summarized by the receiver operating characteristic (ROC) curve and the area under ROC curve. These types of human studies are difficult to arrange, costly, and time consuming. In addition, human observers involved in this type of study should be experts on the image genre to avoid inconsistent scoring through the lengthy study. In two-alternative forced choice (2AFC) studies, known to be faster, two images are compared simultaneously and a single indicator is given. Unfortunately, the 2AFC approach cannot lead to a full ROC curve or a set of image scores. The aim of this work is to propose a methodology in which multiple rounds of the 2AFC studies are used to re-estimate an image confidence score (a.k.a. rating, ranking) and generate the full ROC curve. In the proposed approach, we treat image confidence score as an unknown rating that needs to be estimated and 2AFC as a two-player match game. To achieve this, we use the ELO rating system, which is used for calculating the relative skill levels of players in competitor-versus-competitor games such as chess. The proposed methodology is not limited to ELO, and other rating methods such as TrueSkill™, Chessmetrics, or Glicko can be also used. The presented results, using simulated data, indicate that a full ROC curve can be recovered using several rounds of 2AFC studies and that the best pairing strategy starts with the first round of pairing abnormal versus normal images (as in the classical 2AFC approach) followed by a number of rounds using random pairing. In addition, the proposed method was tested in a pilot human observer study. These pilot results indicate that three to five rounds of 2AFC studies require less human observer time than a full scoring study and that the re-estimated ROC curves and associated area under ROC curve values have high statistical agreement with the full scoring study.

show abstract

Generalization Evaluation of Machine Learning Numerical Observers for Image Quality Assessment

Cited by 36 publications

References 33 publications

CNN as model observer in a liver lesion detection task for x‐ray computed tomography: A phantom study

CNN as model observer in a liver lesion detection task for x‐ray computed tomography: A phantom study

The Relevance Sample-Feature Machine: A Sparse Bayesian Learning Approach to Joint Feature-Sample Selection

Full receiver operating characteristic curve estimation using two alternative forced choice studies

Contact Info

Product

Resources

About