In this paper methods used to measure observer performance are reviewed, and a simple general model for finding and reporting target objects in gray-scale image backgrounds is presented. That model provides the basis for a combined measurement of detection and localization performance in various image-interpretation tasks, whether by human observers or by realized computer algorithms. The model assumes that (1) an observer's detection response and first choice of target location both depend on the "maximally suspicious" finding on an image, (2) a correct (first-choice) localization of the actual target occurs if and only if its location is selected as the most suspicious, and (3) a target's presence does not alter the degree of suspicion engendered by any other (normal) image findings. Formalization of these assumptions relates the ROC curve, which measures the ability to discriminate between images containing targets and images without targets, to the "Localization Response" (LROC) curve, which measures the conjoint ability to detect and correctly localize the actual targets in those images. A maximum-likelihood statistical procedure, developed for a two-parameter "binormal" version of this model, concurrently fits both the ROC and LROC curves from an observer's image ratings and target localizations for a set of image interpretations. The model's application is illustrated (and compared to standard ROC analysis) using sets of rating and localization data from radiologists asked to search chest films for pulmonary nodules. This model is then extended to multiple-report ("free-response") interpretations of multiple-target images, under the stringent requirement that an observer's detection capability and criterion for reporting possible targets both remain stationary across images and across the successive reports made on a given image. That extended model yields formulations and predictions for the so-called "Free-Response" (FROC) curve, and for a recently proposed "Alternative FROC" (AFROC) curve. Tests of that model's "stationarity" assumptions are illustrated using radiologists' free-search interpretations of chest films for pulmonary nodules, and they suggest that human observers may often violate those assumptions when making multiple-report interpretations of images.

No abstract

An "extreme-detector" model for detecting spatially uncertain targets in noisy backgrounds -predicts how both detection and localization abilities are degraded by increasing the number of possible target locations. Experiments 1 and 2 show that the model accurately predicts detection and localization performance in tasks with two, four, and eight locations from d' estimates of the observer's ability to detect the target in a known spatial location. These predictions can be linked to the physical stimuli by combining the extreme-detector model with a "psychophysical" model that specifies how stimulus measures determine the target's detectability in a given location. Single-parameter fits of four such combined models were compared with estimates of detection and localization performance in Experiment 3, which manipulated the target's physical signal-to-noise ratio across various conditions of an eightlocation task. 521Situations that require the detection of visual targets often include considerable uncertainty about the spatial location in which the target might appear. For example, a radiologist who interprets a radiographic image must decide whethe; or not a lesion is present in anyone of many possible anatomic locations. An air-traffic controller who monitors a visual display may need to determine when an aircraft has entered the radar field from any direction. Many of these situations include sources of physical noise that would limit the detectability of the visual signal, even if there were no uncertainty about precisely where the target might appear on the visual display. In such noise-limited situations, any increase in the target's spatial uncertainty will degrade an observer's visual performance, even if there are no time constraints that prevent adequate attention to the information from all relevant spatial locations. Intuitively, the degradation in performance occurs because each additional spatial location that the observer must consider increases his opportunity to encounter a sample of noise extreme enough to resemble an actual target.For a target that may appear in anyone of m distinct spatial locations, an increase in m will reduce both the observer's ability to distinguish cases when the target is present from cases when it is not (target detectability) and his ability to identify the correct location of the target. The following section develops an "extreme-detector" model for performance under spatial uncertainty, which assumes no limitaSupported by USPHS Grant GMI8674, NIH Biomedical Research Support Grant 2-9-9696, and a grant from the W. tion on the observer's ability to process visual information from additional spatial locations. This model predicts a specificrelation between measures of targetdetection and target-localization performance and also predicts how these measures should vary as a function of m, the number of distinct spatial locations in which the target might appear.The three experiments applied the extreme-detector model to observers' performance with noise-limited visual i...