Prevalence effects in newly trained airport checkpoint screeners: Trained observers miss rare targets, too

Abstract

Many socially important search tasks are characterized by low target prevalence, meaning that targets are rarely encountered. For example, transportation security officers (TSOs) at airport checkpoints encounter very few actual threats in carry-on bags. In laboratory-based visual search experiments, low prevalence reduces the probability of detecting targets (Wolfe, Horowitz, & Kenner, 2005). In the lab, this "prevalence effect" is caused by changes in decision and response criteria (Wolfe & Van Wert, 2010) and can be mitigated by presenting a burst of high-prevalence search with feedback (Wolfe et al., 2007). The goal of this study was to see if these effects could be replicated in the field with TSOs. A total of 125 newly trained TSOs participated in one of two experiments as part of their final evaluation following training. They searched for threats in simulated bags across five blocks. The first three blocks were low prevalence (target prevalence ≤ .05) with no feedback; the fourth block was high prevalence (.50) with full feedback; and the final block was, again, low prevalence. We found that newly trained TSOs were better at detecting targets at high compared to low prevalence, replicating the prevalence effect. Furthermore, performance was better (and response criterion was more "liberal") in the low-prevalence block that took place after the high-prevalence block than in the initial three low-prevalence blocks, suggesting that a burst of high-prevalence trials may help alleviate the prevalence effect in the field.

How Do Radiologists Use the Human Search Engine?

“…0.05). Similar results were found with cytologists reading cervical cancer screening slides (61) and with newly trained transportation security officers, looking at luggage X-rays (62) . In all cases, rare targets were missed more often.…”

Improving the Efficacy of Security Screening Tasks: A Review of Visual Search Challenges and Ways to Mitigate Their Adverse Effects

“…Some research suggests that this effect is due to a criterion shift in decision-making (i.e., are searchers more likely to say a target is present or more likely to say a target is absent) as searchers become more biased to miss rather than locate a target when one is present (Godwin et al, 2010;Wolfe et al, 2007; but see also Fleck & Mitroff, 2007). Low target prevalence can be a particularly potent influence, and this effect impacts even professional visual searchers, including radiologists (Evans, Birdwell, & Wolfe, 2013) and airport security screening trainees (Wolfe, Brunelli, Rubinstein, & Horowitz, 2013).…”

The influence of experience upon information-sampling and decision-making behaviour during risk assessment in military personnel

“…However, in X-ray baggage screeners, there is a cost associated with this increase in accuracy: search is more accurate but less rapid for experienced searchers, who approach search in a consistently more cautious manner. This finding suggests that experienced X-ray baggage screeners may be more exhaustive in their searchers (Wolfe, Brunelli, Rubinstein, & Horowitz, 2013), perhaps due to spending longer examining each object or examining more objects (Godwin, Menneer, Cave, Thaibsyah, & Donnelly, 2014).…”