Are retrospective assessments means of people’s experiences? Accounting for interpersonal and intrapersonal variability when comparing retrospective assessment data to ecological momentary assessment data.

Abstract

Retrospective Assessment (RA) scores are often found to be higher than the mean of Ecological Momentary Assessment (EMA) scores about a concurrent period. This difference is generally interpreted as bias towards salient experiences in RA. During RA participants are often asked to summarize their experiences in unspecific terms, leaving room for personal interpretation. As a result, participants may use various strategies to summarize their experiences. In this study, we reanalyzed an existing dataset (N = 92) using a repeated N = 1 approach. We assessed for each participant whether it was likely that their RA score was an approximation of the mean of their experiences as captured by their EMA scores. We found considerable interpersonal differences in the difference between EMA scores and RA scores, as well as some extreme cases. Furthermore, for a considerable part of the sample (n = 46 for positive affect, n = 56 for negative affect), we did not reject the null hypothesis that their RA score represented the mean of their experiences as captured by their EMA scores. We conclude that in its current unspecific form RA may facilitate bias, although not for everyone. Future studies may determine whether differences between RA and EMA are mitigated using more specific forms of RA, while acknowledging interindividual differences.

Measuring Continuous Affect in Daily Life with Intensity Profile Drawings

“…However, a discrete retrospective question, such as "How angry did you feel since the last beep?" suggests that respondents must subjectively summarize their experience, which could lead them to report the average level, the peak intensity, or the most recent instance of anger -leaving it up to the researchers to decide what respondents chose to represent (Leertouwer et al, 2022).…”

Discriminant validity of interval response formats: Investigating the dimensional structure of interval widths

“…For example, respondents might not be able to retrieve the information required to judge the variability of trait-related behaviors and cognition from memory (Ellison et al, 2020;Leertouwer et al, 2022). As a consequence, subjective uncertainty may influence responses to trait questions that are rather intended to measure variability.…”

How to measure and model personality traits in everyday life: A qualitative analysis of 300 big five personality items

“…There are detailed descriptions of the guidelines followed when developing questionnaires (Costa & McCrae, 2008;Goldberg et al, 2006;Hendriks et al, 1999;Hendriks & De Raad, 2002) , sometimes also citing earlier works such as Cattell (1947) lexical scales, but generally there is little to no discussion of why certain items were deemed good indicators of personality or of specific personality traits. Therefore, we want to emphasize that item content, and the various ways in which people engage with it, should receive more attention in research, especially when translating trait constructs into ILD constructs (see also, Leertouwer et al, 2021Leertouwer et al, , 2022.…”

“…When filling out trait questionnaires, individuals are constantly prompted to retrospectively determine whether, or to what degree, they have exhibited such patterns. This mental operation of averaging over one's experiences and behaviors is asking a lot of participants, and may easily involve cognitive biases (Leertouwer et al, 2021(Leertouwer et al, , 2022. As argued above, ILD studies could be an effective tool in assessing these various patterns evoked by personality trait assessments, more directly.…”