powe(R)OC: A power simulation tool for eyewitness lineup ROC analyses

Mah, Eric Y.

doi:10.31234/osf.io/3e4zb

Cited by 5 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These findings highlight the need for researchers to conduct power analyses and make them reproducible just as their data and analyses (Lakens, 2022;Hardwicke et al, 2022). Although power analyses for ROC curve analyses and AUC values can be conducted via the R package "pROC" (Robin et al, 2011), further work is necessary to allow more contextualized power analyses using for example the SESOI (but see Mah, 2022).…”

Section: Smallest Effect Size Of Interest In Eyewitness Memory Researchmentioning

confidence: 99%

The Nature of ROC Practices in Eyewitness Memory Research

Riesthuis,

Otgaar

2024

Preprint

View full text Add to dashboard Cite

Eyewitness memory research has reformed police practices and policy and is sometimes relied upon in legal proceedings. Due to the practical implications derived from this research, it is imperative to evaluate how practical recommendations are postulated. To assess the practical relevance of research, effect sizes and its interpretation play a pivotal role. In this study, we examined how the frequently used effect size Area Under the Curve (AUC) obtained via Receiver Operating Characteristic (ROC) curves are used and interpreted in eyewitness memory research. Via a systematic literature search, we identified 157 eyewitness memory related articles that conducted ROC curve analyses resulting in 1580 AUC values. Approximately 90% of the studies relied on statistical significance to interpret the AUC values. This practice can be problematic as the sample sizes in many studies were rather large which can lead to statistically significant effects that are trivial. Also, more than half of the studies did not report 95% CIs for their AUC values. Finally, power analyses were frequently not conducted and if they were they were, they were oftentimes not reproducible. To improve the practical inferences of eyewitness memory research, we highlight the need of establishing a smallest effect size of interest, focusing on 95% CIs, and conducting reproducible power analyses.

show abstract

Section: Smallest Effect Size Of Interest In Eyewitness Memory Researchmentioning

confidence: 99%

The Nature of ROC Practices in Eyewitness Memory Research

Riesthuis,

Otgaar

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…A power analysis was conducted using the powe(R)OC app in R (Mah, 2022). This analysis indicated that 400 participants are required in order to detect an effect size of 0.15 with approximately 0.8 power.…”

Section: Participantsmentioning

confidence: 99%

Using objective measures to examine the effect of suspect-filler similarity on eyewitness identification performance

McKinley

Peterson

2022

Cogn. Research

View full text Add to dashboard Cite

When selecting fillers to include in a police lineup, one must consider the level of similarity between the suspect and potential fillers. In order to reduce misidentifications, an innocent suspect should not stand out. Therefore, it is important that the fillers share some degree of similarity. Importantly, increasing suspect-filler similarity too much will render the task too difficult reducing correct identifications of a guilty suspect. Determining how much similarity yields optimal identification performance is the focus of the proposed study. Extant research on lineup construction has provided somewhat mixed results. In part, this is likely because similarity is often defined in relative terms due to the subjective nature of similarity. In the current study, we propose an experiment in which we manipulate suspect-filler similarity via a multidimensional scaling model constructed using objective facial measurements. In doing so, we test the “propitious heterogeneity” and the diagnostic-feature-detection hypotheses which predict an advantage of lineups with low similarity fillers in terms of discriminability.

show abstract

“…There is some overlapping functionality between sdtlu and pyWitness, but the main differences are the available models that each package supports, the possibility of extension, and us-ability. Recently an R shiny app, powe(R)OC, has been made available to perform power analyses for eyewitness ROCs (Mah (2022)). These technological solutions will advance the way researchers conduct eyewitness ID analyses.…”

Section: Introductionmentioning

confidence: 99%

pyWitness 1.0: A Python eyewitness identification analysis toolkit

Mickes¹,

Seale-Carlisle²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

pyWitness is a python toolkit for recognition memory experiments, with a focus on eyewitness identification (ID) data analysis and model fitting. The current practice is for researchers to use different statistical packages to analyze a single dataset. pyWitness streamlines the process of the data analysis. In addition to conducting key data analyses (e.g., receiver operating characteristic analysis, confidence accuracy characteristic analysis), statistical comparisons, signal-detection- based model fits, simulated data generation, and power

show abstract

powe(R)OC: A power simulation tool for eyewitness lineup ROC analyses

Cited by 5 publications

References 23 publications

The Nature of ROC Practices in Eyewitness Memory Research

The Nature of ROC Practices in Eyewitness Memory Research

Using objective measures to examine the effect of suspect-filler similarity on eyewitness identification performance

pyWitness 1.0: A Python eyewitness identification analysis toolkit

Contact Info

Product

Resources

About