Monotonicity of rank order probabilities in signal detection models of simultaneous detection and identification

Meyer-Grant, Constantin G.; Klauer, Karl Christoph

doi:10.1016/j.jmp.2021.102615

Cited by 11 publications

(34 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When searching for ways to obtain more informative lineup data, researchers are strongly encouraged to capitalize on SDT’s impressive track record in linking different types of judgments (see Gepshtein et al, 2020 ; Kellen et al, 2021 ; Meyer-Grant & Klauer, 2021 ). For example, Kellen et al ( 2012 ) relied on the close relationship between ranking and yes-no recognition judgments to estimate criterion noise under minimal assumptions (for other examples of joint modeling, see Rouder & Batchelder, 1998 ; Chechile, 2004 , Jang et al, 2009 ).…”

Section: Discussionmentioning

confidence: 99%

Toward a more comprehensive modeling of sequential lineups

Kellen

McAdoo

2022

Cogn. Research

View full text Add to dashboard Cite

Sequential lineups are one of the most commonly used procedures in police departments across the USA. Although this procedure has been the target of much experimental research, there has been comparatively little work formally modeling it, especially the sequential nature of the judgments that it elicits. There are also important gaps in our understanding of how informative different types of judgments can be (binary responses vs. confidence ratings), and the severity of the inferential risks incurred when relying on different aggregate data structures. Couched in a signal detection theory (SDT) framework, the present work directly addresses these issues through a reanalysis of previously published data alongside model simulations. Model comparison results show that SDT modeling can provide elegant characterizations of extant data, despite some discrepancies across studies, which we attempt to address. Additional analyses compare the merits of sequential lineups (with and without a stopping rule) relative to showups and delineate the conditions in which distinct modeling approaches can be informative. Finally, we identify critical issues with the removal of the stopping rule from sequential lineups as an approach to capture within-subject differences and sidestep the risk of aggregation biases.

show abstract

Section: Discussionmentioning

confidence: 99%

Toward a more comprehensive modeling of sequential lineups

Kellen

McAdoo

2022

Cogn. Research

View full text Add to dashboard Cite

show abstract

“…We concede this point for standard old-new recognition, but understanding the differences between different methods for constructing ROCs is important for advanced applications. Full lineup ROCs are a great example that we discuss in detail below, but creating ROCs with a DR criterion could play an important role in other areas as recognition researchers explore tasks that combine multiple types of responses (e.g., Meyer-Grant & Klauer, 2021;Starns, Dube, & Frelinger, 2018) and consider ROCs based on alternative variables, such as response time (e.g., Weidemann & Kahana, 2016).…”

Section: Strength Vs Diagnosticity Ratio Criteriamentioning

confidence: 99%

“…Macmillan & Creelman (2005) refer to such tasks as simultaneous detection and identification tasks. In addition to their longstanding prevalence in eyewitness research, this type of task is beginning to play a role in distinguishing models of recognition memory (e.g., Meyer-Grant & Klauer, 2021). Importantly, researchers have not yet mapped out a theoretically grounded set of guidelines for how ROCs from these tasks should be created and interpreted.…”

Section: Lineup Rocsmentioning

confidence: 99%

“…When this is the case, it demonstrates that the decision rule used to construct the ROC achieves optimal performance and bears a monotonic relationship with the DR (e.g., Figure 2). Critically, the DR strategy can be applied even when there is no clear method for creating a strength-based ROC; thus, it provides an important tool as memory researchers begin to investigate more complex recognition tasks (Meyer-Grant & Klauer, 2021;Smith et al, 2020;Starns et al, 2018).…”

Section: Summary and Recommendationsmentioning

confidence: 99%

See 1 more Smart Citation

A theory-based approach for constructing recognition Receiver Operating Characteristics (ROCs) in complex tasks, with an application to full lineup ROCs

Starns¹,

Cohen²,

Tuttle³

2022

Preprint

View full text Add to dashboard Cite

Recognition memory researchers are extending Receiver Operating Characteristic (ROC) methods to complex tasks that require new analysis tools and a deeper theoretical understanding of the principles of ROC construction and interpretation. ROCs from an eyewitness lineup task are a great example of this trend. The current paper establishes some key theoretical principles relevant to lineup ROC research and applies these principles to establish a set of guidelines for memory ROCs. Theoretical demonstrations show that creating ROCs based on likelihood or diagnosticity ratios (DRs) provides a general method for tasks that are not amendable to traditional methods. The demonstrations also establish an alternative interpretation for area under the curve (AUC) for an ROC that connects the observed operating points with straight lines, reinforcing the value of this measure for eyewitness research and revealing other potential applications. Although DR-based ROCs are theoretically justified, parameter recovery simulations show that creating lineup ROCs with observed sample DRs leads to biased estimation of AUC. We propose a new ROC method to correct for this bias, whereby the observed data are fit with a decision model, and ROCs are constructed based on the DRs from the model. Finally, we show how a DR-based ROC can be used to generate a full theoretical ROC for lineup tasks, and note that these theoretical curves reveal a previously unappreciated property of lineup decision making: changing response conservativeness changes the evidence distributions that determine lineup responses.

show abstract

“…Often, both X o and X n are assumed to follow a normal distribution (i.e., X o ∼ N (μ X o , σ X o ) and X n ∼ N (μ X n , σ X n )), leading to the so-called Gaussian SDT model sub-class. 1 In recent years, however, more complex experimental paradigms have come into focus as they allow for novel and more refined tests of specific model assumptions (e.g., Kellen et al, 2021;Kellen & Klauer, 2014;Meyer-Grant & Klauer, 2021;Voormann et al, 2021). Consider, for example, the situation in which participants are presented with a set of m > 1 stimuli at the same time in each test trial.…”

Section: Introductionmentioning

confidence: 99%

Disentangling Different Aspects of Between-Item Similarity Unveils Evidence Against the Ensemble Model of Lineup Memory

Meyer-Grant

Klauer

2022

Comput Brain Behav

Self Cite

View full text Add to dashboard Cite

For modeling recognition decisions in a typical eyewitness identification lineup task with multiple simultaneously presented test stimuli (also known as simultaneous detection and identification), essentially two different models based on signal detection theory are currently under consideration. These two models mainly differ with respect to their assumptions regarding the interplay between the memory signals of different stimuli presented in the same lineup. The independent observations model (IOM), on the one hand, assumes that the memory signal of each simultaneously presented test stimulus is separately assessed by the decision-maker, whereas the ensemble model (EM), on the other hand, assumes that each of these memory signals is first compared with and then assessed relative to its respective context (i.e., the memory signals of the other stimuli within the same lineup). Here, we discuss some reasons why comparing confidence ratings between trials with and without a dud (i.e., a lure with no systematic resemblance to the target) in an otherwise fair lineup—results of which have been interpreted as evidence in favor of the EM—is in fact inconclusive for differentiating between the EM and the IOM. However, the lack of diagnostic value hinges on the fact that in these experiments two aspects of between-item similarity (viz. old–new and within-lineup similarity) are perfectly confounded. Indeed, if separately manipulating old–new similarity, we demonstrate that EM and IOM make distinct predictions. Following this, we show that previously published data are inconsistent with the predictions made by the EM.

show abstract

Monotonicity of rank order probabilities in signal detection models of simultaneous detection and identification

Cited by 11 publications

References 74 publications

Toward a more comprehensive modeling of sequential lineups

Toward a more comprehensive modeling of sequential lineups

A theory-based approach for constructing recognition Receiver Operating Characteristics (ROCs) in complex tasks, with an application to full lineup ROCs

Disentangling Different Aspects of Between-Item Similarity Unveils Evidence Against the Ensemble Model of Lineup Memory

Contact Info

Product

Resources

About