Spatial Localization Accuracy of Radiologists in Free-Response Studies: Inferring Perceptual FROC Curves from Mark-Rating Data

Chakraborty, Dev P.; Yoon, Hong-Jun; Mello‐Thoms, Claudia

doi:10.1016/j.acra.2006.10.015

Cited by 29 publications

(21 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The analysis was performed using JAFROC1 analysis software version 1.0 (http:// devchakraborty.com/) [19]; figure-of-merit values representing the diagnostic accuracy were compared in the non-enhanced and the gadoxetic acid-enhanced image sets for both T 2 weighted and DW imaging using the F-test. To evaluate the effect of gadoxetic acid enhancement on lesion characterisation, we identified focal hepatic lesions that were detected on both nonenhanced and gadoxetic acid-enhanced images by each of the two readers.…”

Section: Discussionmentioning

confidence: 99%

The effect of gadoxetic acid enhancement on lesion detection and characterisation usingT₂weighted imaging and diffusion weighted imaging of the liver

Ss²,

Ih³

et al. 2012

BJR

View full text Add to dashboard Cite

Objectives: To evaluate the effect of gadoxetic acid enhancement on the detection and characterisation of focal hepatic lesions on T 2 weighted and diffusion weighted (DW) images. Methods: A total of 63 consecutive patients underwent T 2 weighted and DW imaging before and after gadoxetic acid enhancement. Two blinded readers independently identified all of the focal lesions using a five-point confidence scale and characterised each lesion using a three-point scale: 1, non-solid; 2, indeterminate; and 3, solid. For both T 2 weighted and DW imaging, the accuracies for detecting focal lesions were compared using the free-response receiver operating characteristic analysis; the accuracies for lesion characterisation were compared using the McNemar test between non-enhanced and gadoxetic acid-enhanced image sets. For hepatic lesions >1 cm, the lesion-to-liver contrast-to-noise ratio (CNR) and the apparent diffusion coefficient (ADC) were compared in the non-enhanced and enhanced image sets using the generalised estimating equations. Results: For both T 2 weighted and DW images, the accuracies for detecting focal lesions (p>0.52) and those for lesion characterisation (p>0.63) did not differ significantly between the non-enhanced and enhanced image sets. The lesion-to-liver CNR was significantly higher on enhanced DW images than on non-enhanced DW images (p50.02), although the difference was not significant for T 2 weighted imaging (p50.65). The mean ADC values of lesions did not differ significantly on enhanced and non-enhanced DW imaging (p50.75). Conclusion: The acquisition of T 2 weighted and DW images after administration of gadoxetic acid has no significant effect on the detection or characterisation of focal hepatic lesions, although it improves the lesion-to-liver CNR on DW images. Various contrast agents have been developed and utilised for MRI of the liver in order to facilitate the detection and characterisation of focal hepatic lesions. Gadoxetic acid (gadolinium-ethoxybenzyl-diethylenetriamine pentaacetic acid, PrimovistH; Bayer Schering Pharma, Berlin, Germany) is a recently developed, liverspecific contrast agent. As it has combined extracellular and hepatocyte-specific properties, gadoxetic acid can provide functional information regarding the cellular composition of focal hepatic lesions on hepatobiliary phase imaging as well as haemodynamic information on dynamic MRI following bolus injection. These properties of gadoxetic acid have been reported to improve the accuracy of liver MRI for lesion detection and characterisation [1][2][3][4][5][6][7].By contrast, these advantages of gadoxetic acidenhanced liver MRI are obtained with increased examination time, as delayed scanning approximately 20 min after contrast administration is necessary for optimal hepatobiliary phase imaging [4,5,[7][8][9]. Among the pulse sequences commonly acquired for clinical liver MRI, T 2 weighted and diffusion weighted (DW) imaging are frequently performed using a respiratory-triggered method in order to improve image quality...

show abstract

Section: Discussionmentioning

confidence: 99%

The effect of gadoxetic acid enhancement on lesion detection and characterisation usingT₂weighted imaging and diffusion weighted imaging of the liver

Ss²,

Ih³

et al. 2012

BJR

View full text Add to dashboard Cite

show abstract

“…Harisinghani et al [17], studying digital workstation ergonomics, have suggested that the distance from the observer to the screen should be at least 25 inches (63.5 cm) [17]. Other approaches to scoring that do not rely on an acceptance radius include using statistical methods to classify ''perceptual hits'' and ''perceptual misses'' [9] or asking readers to outline a lesion [18,19].…”

Section: Discussionmentioning

confidence: 99%

“…If the mark falls within the circle whose centre corresponds to the centre of the lesion as predetermined by the investigator and whose radius is the acceptance radius (the acceptance circle), the mark is scored as an LL. Any mark outside of the acceptance circle is scored as an NLL [9]. The purpose of this study was to evaluate the effects of varying the acceptance radius for a nodule detection task in chest radiography in order to suggest guidelines for determining the acceptance radius.…”

mentioning

confidence: 99%

On the choice of acceptance radius in free-response observer performance studies

Haygood

Ryan²,

Brennan³

et al. 2013

BJR

View full text Add to dashboard Cite

Objectives: Choosing an acceptance radius or proximity criterion is necessary to analyse free-response receiver operating characteristic (FROC) observer performance data. This is currently subjective, with little guidance in the literature about what is an appropriate acceptance radius. We evaluated varying acceptance radii in a nodule detection task in chest radiography and suggest guidelines for determining an acceptance radius. Methods: 80 chest radiographs were chosen, half of which contained nodules. We determined each nodule's centre. 21 radiologists read the images. We created acceptance radii bins of ,5 pixels, ,10 pixels, ,20 pixels and onwards up to ,200 and 200+ pixels. We counted lesion localisations in each bin and visually compared marks with the borders of nodules. Results: Most reader marks were tightly clustered around nodule centres, with tighter clustering for smaller than for larger nodules. At least 70% of readers' marks were placed within ,10 pixels for small nodules, ,20 pixels for medium nodules and ,30 pixels for large nodules. Of 72 inspected marks that were less than 50 pixels from the centre of a nodule, only 1 fell outside the border of a nodule. Conclusion: The acceptance radius should be based on the larger nodule sizes. For our data, an acceptance radius of 50 pixels would have captured all but 2 reader marks within the borders of a nodule, while excluding only 1 true-positive mark. The choice of an acceptance radius for FROC analysis of observer performance studies should be based on the size of larger abnormalities. Observer performance studies are often used to evaluate imaging systems in medicine. These studies are usually organised so that observers search for a particular abnormality in a set of images, indicate whether the abnormality is present and then do the same thing again on another occasion under different circumstances. There is a variety of ways in which to judge the accuracy of the observers' responses. Perhaps the most widely used method is one in which observers indicate the presence or absence of the searched-for lesion and the level of confidence with which they have identified or excluded the lesion. From this information, a receiver operating characteristic (ROC) curve is generated [1][2][3][4][5]. The free-response ROC (FROC) method is an alternative approach to analysis that uses lesion location information, and thus more closely mimicks those actual clinical practices that involve a challenging search and yields a higher statistical power than the ROC method [6,7]. Free-response methods also allow separate analysis of success in finding more than one abnormality per image [7]. In the free-response method, the observer locates each lesion, marks it and assigns a confidence rating to each marked lesion. This method is intended to avoid counting a response as correct in situations in which the observer, although correctly indicating the presence of an abnormality in an image actually containing the abnormality, was led astray by a false-positive and did not ...

show abstract

“…Acceptance regions of lesions are varied in peer review literature, this study has followed the Chakraboty 34 recommendation of a maximum diameter of a lesion or 3 mm. Acceptance radiuses are a controversial topic in observer performance studies and Chakraborty 34,35 considers the question of what is the maximum inaccuracy with no clinical impact that would be acceptable in a study scenario.…”

Section: Methodsmentioning

confidence: 99%

Intraorbital foreign body detection and localisation by radiographers: A preliminary JAFROC observer performance study

et al. 2016

View full text Add to dashboard Cite

show abstract

Spatial Localization Accuracy of Radiologists in Free-Response Studies: Inferring Perceptual FROC Curves from Mark-Rating Data

Cited by 29 publications

References 32 publications

The effect of gadoxetic acid enhancement on lesion detection and characterisation usingT₂weighted imaging and diffusion weighted imaging of the liver

The effect of gadoxetic acid enhancement on lesion detection and characterisation usingT₂weighted imaging and diffusion weighted imaging of the liver

On the choice of acceptance radius in free-response observer performance studies

Intraorbital foreign body detection and localisation by radiographers: A preliminary JAFROC observer performance study

Contact Info

Product

Resources

About

Spatial Localization Accuracy of Radiologists in Free-Response Studies: Inferring Perceptual FROC Curves from Mark-Rating Data

Cited by 29 publications

References 32 publications

The effect of gadoxetic acid enhancement on lesion detection and characterisation usingT2weighted imaging and diffusion weighted imaging of the liver

The effect of gadoxetic acid enhancement on lesion detection and characterisation usingT2weighted imaging and diffusion weighted imaging of the liver

On the choice of acceptance radius in free-response observer performance studies

Intraorbital foreign body detection and localisation by radiographers: A preliminary JAFROC observer performance study

Contact Info

Product

Resources

About

The effect of gadoxetic acid enhancement on lesion detection and characterisation usingT₂weighted imaging and diffusion weighted imaging of the liver

The effect of gadoxetic acid enhancement on lesion detection and characterisation usingT₂weighted imaging and diffusion weighted imaging of the liver