Computational assessment of visual search strategies in volumetric medical images

Abstract

Abstract. When searching through volumetric images [e.g., computed tomography (CT)], radiologists appear to use two different search strategies: "drilling" (restrict eye movements to a small region of the image while quickly scrolling through slices), or "scanning" (search over large areas at a given depth before moving on to the next slice). To computationally identify the type of image information that is used in these two strategies, 23 naïve observers were instructed with either "drilling" or "scanning" when searching for target T's in 20 volumes of faux lung CTs. We computed saliency maps using both classical two-dimensional (2-D) saliency, and a three-dimensional (3-D) dynamic saliency that captures the characteristics of scrolling through slices. Comparing observers' gaze distributions with the saliency maps showed that search strategy alters the type of saliency that attracts fixations. Drillers' fixations aligned better with dynamic saliency and scanners with 2-D saliency. The computed saliency was greater for detected targets than for missed targets. Similar results were observed in data from 19 radiologists who searched five stacks of clinical chest CTs for lung nodules. Dynamic saliency may be superior to the 2-D saliency for detecting targets embedded in volumetric images, and thus "drilling" may be more efficient than "scanning."

Eye-Tracking in the Study of Visual Expertise: Methodology and Approaches in Medicine

“…Several recent studies provide evidence that radiologists examining CT images in three-dimensions developed visual patterns that involved maintaining little movement in one dimension while scanning across the plane of the other two dimensions (Drew, Vo, Olwal, et al, 2013;Wen et al, 2016). Two distinct techniques within this pattern could be clustered, but no significant superiority of one technique over another was demonstrated.…”

“…Wolfe et al (1994) summarize this process in the "Guided Search Model," in which bottom-up attention to distracters is at least partially suppressed by the top-down effects of visual expertise. The expert's knowledge provides scene guidance to relevant parts of the image, which combines with these effects to create a mental representation of the likely location of targets (Wen et al, 2016). This course of attentional gaze may be modulated by systematic training algorithms for visual inspection (for example, a trained pattern of attention to each potential diagnostic target within a chest radiograph), as well as clinical information (Draschkow et al, 2014;Drew, Evans, et al, 2013;Wolfe et al, 2015;Wolfe et al, 2007).…”

“…Eye-tracking has been utilized to better understand the development of a "priority map," within a radiological image, and how this priority map may evolve during the diagnostic process. It is proposed that the priority map must change as the radiologist's eyes move about the image as the salience of items will change with their distance from the present fixation (Wen et al, 2016). Interestingly, the development of visual expertise with training in radiology seems to indicate a move towards efficiency, and away from repeated attention to "non-priority" regions of the diagnostic image (G. Tourassi et al, 2013;Wolfe et al, 2015).…”

“…At this level of visual processing, the radiologist guides attention toward a mental representation of the features and potential location of a target (Wen et al, 2016). Top-down processing is also employed at the point of diagnosis, when the visual item is matched to both a mental depiction of the target, and general medical knowledge (Drew, Evans, et al, 2013;Drew, Vo, Olwal, et al, 2013).…”

“…D. Tourassi et al, 2010; | F L R 46 2015). The visual expertise learned by radiologists is an example of a search-related task, in which the radiologist identifies visual "targets" in an image containing both expected and "distractor" elements (Wen et al, 2016;Wolfe et al, 2015). Screening tests, both in radiology and pathology, are generally performed using a visual search model (Stewart et al, 2007;Wolfe, 1995).…”