A theory-based approach for constructing recognition Receiver Operating Characteristics (ROCs) in complex tasks, with an application to full lineup ROCs

Abstract

Recognition memory researchers are extending Receiver Operating Characteristic (ROC) methods to complex tasks that require new analysis tools and a deeper theoretical understanding of the principles of ROC construction and interpretation. ROCs from an eyewitness lineup task are a great example of this trend. The current paper establishes some key theoretical principles relevant to lineup ROC research and applies these principles to establish a set of guidelines for memory ROCs. Theoretical demonstrations show that creating ROCs based on likelihood or diagnosticity ratios (DRs) provides a general method for tasks that are not amendable to traditional methods. The demonstrations also establish an alternative interpretation for area under the curve (AUC) for an ROC that connects the observed operating points with straight lines, reinforcing the value of this measure for eyewitness research and revealing other potential applications. Although DR-based ROCs are theoretically justified, parameter recovery simulations show that creating lineup ROCs with observed sample DRs leads to biased estimation of AUC. We propose a new ROC method to correct for this bias, whereby the observed data are fit with a decision model, and ROCs are constructed based on the DRs from the model. Finally, we show how a DR-based ROC can be used to generate a full theoretical ROC for lineup tasks, and note that these theoretical curves reveal a previously unappreciated property of lineup decision making: changing response conservativeness changes the evidence distributions that determine lineup responses.