&lt;title&gt;A Free Response Approach To The Measurement And Characterization Of Radiographic Observer Performance&lt;/title&gt;

Bunch, Philip C.; Hamilton, J. F.; Sanderson, G.; Simmons, Arthur H.

doi:10.1117/12.955926

Cited by 191 publications

(152 citation statements)

References 0 publications

Supporting

Mentioning

152

Contrasting

Order By: Relevance

“…As ζ decreases from ∞ to −∞, y(ζ) increases from 0 to y max (y max ≤ 1) and x(ζ) increases from 0 to x max . These are common characteristics of observed FROC curves (2,3,5,(20)(21)(22). The end-point (x max , y max ) is reached when all marked regions are counted.…”

Section: Introductionmentioning

confidence: 80%

Spatial Localization Accuracy of Radiologists in Free-Response Studies: Inferring Perceptual FROC Curves from Mark-Rating Data

2007

View full text Add to dashboard Cite

Free-response data consists of a set of mark-ratings pairs. Prior to analysis the data is classified or "scored" into lesion and non-lesion localizations. The scoring is done by choosing an acceptanceradius and classifying marks within the acceptance-radius of lesion centers as lesion localizations, and all other marks are classified as non-lesion localizations. The scored data is plotted as a freeresponse receiver operating characteristic (FROC) curve, essentially a plot of appropriately normalized numbers of lesion localizations vs. non-lesion localizations. Scored FROC curves are frequently used to compare imaging systems and computer aided detection (CAD) algorithms. However, the choice of acceptance-radius is arbitrary. This makes it difficult to compare curves from different studies and to estimate true performance. To resolve this issue the concept of two types of marks is introduced: perceptual hits and perceptual misses. A perceptual hit is a mark made in response to the observer seeing the lesion. A perceptual miss is a mark made in response to the observer seeing a (lesion-like) non-lesion. A method of estimating the most probable numbers of perceptual hits and misses is described. This allows one to plot a perceptual FROC operating point and by extension a perceptual FROC curve. Unlike a scored FROC operating point, a perceptual point is independent of the choice of acceptance-radius. The method does not allow one to identify individual marks as perceptual hits or misses -only the most probable numbers. It is based on a 3-parameter statistical model of the spatial distributions of perceptual hits and misses relative to lesion centers. The method has been applied to an observer dataset in which mammographers, and residents with different levels of experience were asked to locate lesions in mammograms. The perceptual operating points suggest superior performance for the mammographers and equivalent performance for residents in the first and second mammography rotations. These results and the model validation are preliminary as they are based on a small dataset. The significance of this study is showing that it is possible to probabilistically determine if a mark resulted from seeing a lesion or a nonlesion. Using the method developed in this study one could perform acceptance-radius independent estimation of observer performance.

show abstract

Section: Introductionmentioning

confidence: 80%

Spatial Localization Accuracy of Radiologists in Free-Response Studies: Inferring Perceptual FROC Curves from Mark-Rating Data

2007

View full text Add to dashboard Cite

show abstract

“…Fig. 1(a) shows the free-response receiver operating characteristic (FROC) analysis [17] of the implemented algorithms on the entire dataset. The global Sensitivity is evaluated as ❚ P ❚ P ✰❋◆ on each image and then averaged together.…”

Section: Resultsmentioning

confidence: 99%

Automatic retina exudates segmentation without a manually labelled training set

Giancardo¹,

Mériaudeau²,

Karnowski³

et al. 2011

2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro

View full text Add to dashboard Cite

Diabetic macular edema (DME) is a common vision threatening complication of diabetic retinopathy. In a large scale screening environment DME can be assessed by detecting exudates (a type of bright lesions) in fundus images. In this work, two new methods for the detection of exudates are presented. The methods do not require a lesion training set so the need to ground-truth data is avoided with significant time savings and independence from human error. We evaluate our algorithm with a new publicly available dataset from various ethnic groups and levels of DME. Also, we compare our results with two recent exudate segmentation algorithms on the same dataset. In all of our tests, our algorithms are either outperforming or in line with existing methods. Additionally, the computational time is one order of magnitude less than similar methods.

show abstract

“…Assuming that the images are large compared with the signal size and that they occur independently, we can use a Poisson model 6 for the probability of having in an image k false signals with values in D:…”

Section: Iib the Noise Modelmentioning

confidence: 99%

“…The additional test in which all features with s Ͼ s d are declared as positive is described by the free response operating characteristic ͑FROC͒. 6 In order to study signal detectability using this decision mechanism, we need to transform the signal distribution f and the false signal density as functions of s. For each s value, we can define a domain D s ϵ͕z ͉ ␥͑z͒ ജ s͖. For each domain D s , we have a fraction ͑s͒ of signals, and an average number ͑s͒ of false signals, with z values satisfying ␥͑z͒ ജ s. In this manner, we have defined and as a function of signal specificity score s.…”

Section: Iie Detectability Evaluation Using the Signal Specificity mentioning

confidence: 99%

“…5 As precursors of this work we cite Bunch et al 6 who originally proposed a Poisson statistic for the number of false signals appearing in an image background in their free response operating characteristic ͑FROC͒ model, and the works of Swensson, 7-9 who proposed decision mechanisms based on the score of the most suspicious location appearing in an image as the basis of a receiver operating characteristic ͑ROC͒ and a localization ROC ͑LROC͒ model. Here, we will describe a generalization to the case when the observed suspicious locations are characterized by multidimensional variables, and we show alternative decision mechanisms that use the likelihoods of the ensemble of suspicious locations observed in an image.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Model for the detection of signals in images with multiple suspicious locations

Popescu

2008

Medical Physics

View full text Add to dashboard Cite

A signal detection model is presented that combines a signal model and a noise model providing mathematical descriptions of the frequency of appearance of the signals, and of the signal-like features naturally occurring in the background. We derive expressions for the likelihood functions for the whole ensemble of observed suspicious locations, in various possible combinations of signals and false signal candidates. As a result, this formalism is able to describe several new types of detection tests using likelihood ratio statistics. We have a global image abnormality test and an individual signal detection test. The model also provides an alternative mechanism in which is selected the combination of signal and noise features candidates that has the maximum likelihood. These tests can be analyzed with a variety of operating characteristic curves ͑ROC, LROC, FROC, etc.͒. In the mathematical formalism of the model, all the details characterizing the suspicious features are reduced to a single scalar function, which we name the signal specificity function, representing the frequency that a signal takes a certain value relative to the frequency of having a false signal with the same value in an image of given size. The signal specificity function ranks the degree of suspiciousness of the features found, and can be used to unify into a single score all the suspicious feature characteristics, and then apply the usual decision conventions as in the Swensson's detection model ͓Med. Phys. 23, 1709-1725 ͑1996͔͒. We present several examples in which these tests are compared. We also show how the signal specificity function can be used to model various degrees of accuracy of the observer's knowledge about image noise and signal statistical properties. Aspects concerning modeling of the human observer are also discussed.

show abstract

<title>A Free Response Approach To The Measurement And Characterization Of Radiographic Observer Performance</title>

Cited by 191 publications

References 0 publications

Spatial Localization Accuracy of Radiologists in Free-Response Studies: Inferring Perceptual FROC Curves from Mark-Rating Data

Spatial Localization Accuracy of Radiologists in Free-Response Studies: Inferring Perceptual FROC Curves from Mark-Rating Data

Automatic retina exudates segmentation without a manually labelled training set

Model for the detection of signals in images with multiple suspicious locations

Contact Info

Product

Resources

About