Navigating the manifold of skin conductance response quantification approaches – a direct comparison of Trough-to-Peak, Baseline-correction and model-based approaches in Ledalab and PsPM

Abstract

Scientific work rests fundamentally upon data, their measurement, processing, analysis, illustration and interpretation. Raw data have to be processed to be ready for statistical analyses and interpretation. While these processing pipelines can be well defined and standardized, they are often characterized by substantial heterogeneity. Here, we present results from a systematic literature search on the different SCR response quantification approaches used in the literature by using fear conditioning research as a case example. Next, we applied seven of the identified approaches (trough-to-peak scoring, script-based baseline-correction, Ledalab as well as four different models implemented in the software PsPM) to two fear conditioning datasets differing in key procedural specifications (i.e., CS duration, reinforcement rate, number of trials). This can be viewed as a set of robustness analyses (i.e., same data subjected to different methods) aiming to investigate if and to what extent these methods yield comparable results. To our knowledge, no formal framework for the evaluation of robustness analyses exists to date, but we may borrow some criteria from a framework suggested for the evaluation of ‘replicability’ in general. Our results from seven different SCR response quantification approaches applied to two datasets suggest that there may be no single approach that consistently yields larger effect sizes across both datasets. Yet, at least some of the approaches employed show consistent effect sizes within each dataset indicating comparability. Finally, we highlight substantial heterogeneity also within most quantification approaches and discuss implications and potential remedies.

A data multiverse analysis investigating non-model based SCR quantification approaches

“…As a consequence, as already argued in 1971, it is unclear to what extent the interpretation of any single set of SCR results hinges on the specific choices for SCR quantification. Unfortunately, little systematic comparative work exists to date (e.g., Green et al, 2014, Kuhn et al, 2021.…”

“…al., 2014) or PsPM (formerly SCRalyze; , Bach et al, 2010. Kuhn et al, (2021) provide a summary and an overview and independent comparison of these comparative studies. Conclusions drawn from these studies are as heterogeneous as the approaches themselves and it seems unlikely that one specific approach can be considered to be generally 'superior' over the others.…”

“…Conclusions drawn from these studies are as heterogeneous as the approaches themselves and it seems unlikely that one specific approach can be considered to be generally 'superior' over the others. Group-level trial-by-trial agreement across different approaches was fair to substantial with some evidence for stimulus-type dependent agreement rates (Kuhn et al, 2021). It is however likely that different approaches may be more or less suitable under specific conditions including stimulus duration, reinforcement rate, and stimulus sequence timings (Green et al 2014, Kuhn et al, 2021, Bach & Melinsczak, 2020.…”

“…Group-level trial-by-trial agreement across different approaches was fair to substantial with some evidence for stimulus-type dependent agreement rates (Kuhn et al, 2021). It is however likely that different approaches may be more or less suitable under specific conditions including stimulus duration, reinforcement rate, and stimulus sequence timings (Green et al 2014, Kuhn et al, 2021, Bach & Melinsczak, 2020. Furthermore, different parameters are used to define time windows with distinct specifications in the different response quantification approaches resulting in substantial heterogeneity (as summarized by Pineles et al, 2009).…”

“…Of note, also the data multiverse inevitably implies a multiverse of statistical results given that the same statistical models are applied to differently processed data, that however are derived from identical raw data. This has been highlighted for psychology in general (Silberzahn et al, 2018;Steegen et al, 2016) and for fear conditioning research in particular (Lonsdorf, Klingelhöfer-Jens, et al, 2019;Lonsdorf, Merz, et al, 2019, Lonsdorf et al, 2021. To this end, multiversestyle data analyses can be considered a large set of systematic robustness analyses that apply different (equally justifiable) processing pipelines to the same dataset.…”