Evaluation of the channelized Hotelling observer with an internal-noise model in a train-test paradigm for cardiac SPECT defect detection

Brankov, Jovan G.

doi:10.1088/0031-9155/58/20/7159

Cited by 40 publications

(34 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In separate preliminary studies [11]-[13] we also implemented CHO with Gabor filters, DOG, SDOG and LG channels as well as eleven internal noise models and we noted that the accuracy in any combination was lower than proposed CSVM and MKCRVM. Analysis of all possible combination is out of the scope of this work and is in considerations for a separate publication.…”

Section: Resultsmentioning

confidence: 99%

“…The internal noise model is selected and tuned to match the performance of a given observer. Different types of internal noise models have also been proposed (see [11]-[13] for a review). Alternative numerical observers are, for example, proposed in [14]-[17] based on human linear template estimation; in [18] and [19] efforts are made to incorporate nonlinearity into models.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generalization Evaluation of Machine Learning Numerical Observers for Image Quality Assessment

Kalayeh

Marin

Brankov

2013

IEEE Trans. Nucl. Sci.

View full text Add to dashboard Cite

In this paper, we present two new numerical observers (NO) based on machine learning for image quality assessment. The proposed NOs aim to predict human observer performance in a cardiac perfusion-defect detection task for single-photon emission computed tomography (SPECT) images. Human observer (HumO) studies are now considered to be the gold standard for task-based evaluation of medical images. However such studies are impractical for use in early stages of development for imaging devices and algorithms, because they require extensive involvement of trained human observers who must evaluate a large number of images. To address this problem, numerical observers (also called model observers) have been developed as a surrogate for human observers. The channelized Hotelling observer (CHO), with or without internal noise model, is currently the most widely used NO of this kind. In our previous work we argued that development of a NO model to predict human observers' performance can be viewed as a machine learning (or system identification) problem. This consideration led us to develop a channelized support vector machine (CSVM) observer, a kernel-based regression model that greatly outperformed the popular and widely used CHO. This was especially evident when the numerical observers were evaluated in terms of generalization performance. To evaluate generalization we used a typical situation for the practical use of a numerical observer: after optimizing the NO (which for a CHO might consist of adjusting the internal noise model) based upon a broad set of reconstructed images, we tested it on a broad (but different) set of images obtained by a different reconstruction method. In this manuscript we aim to evaluate two new regression models that achieve accuracy higher than the CHO and comparable to our earlier CSVM method, while dramatically reducing model complexity and computation time. The new models are defined in a Bayesian machine-learning framework: a channelized relevance vector machine (CRVM) and a multi-kernel CRVM (MKCRVM).

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Generalization Evaluation of Machine Learning Numerical Observers for Image Quality Assessment

Kalayeh

Marin

Brankov

2013

IEEE Trans. Nucl. Sci.

View full text Add to dashboard Cite

show abstract

“…Thus, we used an ensemble size of 2000 samples/class as our gold standard and we did not model internal noise in the observer. To match the human observer performance, internal noise can be added (Brankov, 2013). The work presented in this paper could be extended to include internal noise in the test statistics calculations.…”

Section: Discussionmentioning

confidence: 99%

A comparison of resampling schemes for estimating model observer performance with small ensembles

et al. 2017

View full text Add to dashboard Cite

In objective assessment of image quality, an ensemble of images is used to compute the 1st and 2nd order statistics of the data. Often, only a finite number of images is available, leading to the issue of statistical variability in numerical observer performance. Resampling-based strategies can help overcome this issue. In this paper, we compared different combinations of resampling schemes (the leave-one-out (LOO) and the half-train/half-test (HT/HT)) and model observers (the conventional channelized Hotelling observer (CHO), channelized Linear discriminant (CLD) and channelized Quadratic discriminant (CQD)). Observer performance was quantified by the area under the ROC curve (AUC). For a binary classification task and for each observer, the AUC value for an ensemble size of 2000 samples per class served as a gold standard for that observer. Results indicated that each observer yielded different performance depending on the ensemble size and the resampling scheme. For a small ensemble size, the combination [CHO, HT/HT] had more accurate rankings than the combination [CHO, LOO]. Using the LOO scheme, the CLD and CHO had similar performance for large ensembles. However, the CLD outperformed the CHO and gave more accurate rankings for smaller ensembles. As the ensemble size decreased, the performance of [CHO, LOO] combination was seriously deteriorated as opposed to the [CLD, LOO] combination. Thus, it might be desirable to use the CLD with the LOO scheme when smaller ensemble size is available.

show abstract

“…For human observers, re-scoring of the four images in a 4AFC test could result in a different output (percentage correct value) due to variations in neural firing, intrinsic inconsistency in receptor response, and loss of information during neural transmission. 18 However, using the MO to re-score a given set of 4AFC images will produce each time the same score, within a specific confidence interval. This makes the introduction of the internal noise a reasonable step when trying to model human observer data and its associated variabilities.…”

Section: Discussionmentioning

confidence: 99%

Development and application of a channelized Hotelling observer for DBT optimization on structured background test images with mass simulating targets

et al. 2016

View full text Add to dashboard Cite

Digital breast tomosynthesis (DBT) is a 3D mammography technique that promises better visualization of low contrast lesions than conventional 2D mammography. A wide range of parameters influence the diagnostic information in DBT images and a systematic means of DBT system optimization is needed. The gold standard for image quality assessment is to perform a human observer experiment with experienced readers. Using human observers for optimization is time consuming and not feasible for the large parameter space of DBT. Our goal was to develop a model observer (MO) that can predict human reading performance for standard detection tasks of target objects within a structured phantom and subsequently apply it in a first comparative study. The phantom consists of an acrylic semi-cylindrical container with acrylic spheres of different sizes and the remaining space filled with water. Three types of lesions were included: 3D printed spiculated and non-spiculated mass lesions along with calcification groups. The images of the two mass lesion types were reconstructed with 3 different reconstruction methods (FBP, FBP with SRSAR, MLTR pr ) and read by human readers. A Channelized Hotelling model observer was created for the non-spiculated lesion detection task using five Laguerre-Gauss channels, tuned for better performance. For the non-spiculated mass lesions a linear relation between the MO and human observer results was found, with correlation coefficients of 0.956 for standard FBP, 0.998 for FBP with SRSAR and 0.940 for MLTR pr . Both the MO and human observer percentage correct results for the spiculated masses were close to 100%, and showed no difference from each other for every reconstruction algorithm.

show abstract

Evaluation of the channelized Hotelling observer with an internal-noise model in a train-test paradigm for cardiac SPECT defect detection

Cited by 40 publications

References 52 publications

Generalization Evaluation of Machine Learning Numerical Observers for Image Quality Assessment

Generalization Evaluation of Machine Learning Numerical Observers for Image Quality Assessment

A comparison of resampling schemes for estimating model observer performance with small ensembles

Development and application of a channelized Hotelling observer for DBT optimization on structured background test images with mass simulating targets

Contact Info

Product

Resources

About