Do people agree on how they and others are acting? Examining the degree of target–observer and observer–observer agreement about current behavior as it changes across situations.

Abstract

The purpose of the present research was to test the level of agreement between targets and observers both at any given moment and as the targets’ current behavior (assessed as personality states) change across moments. Ninety-seven target participants participated in 22 different activities across 20 1-hour long sessions in a laboratory setting while reporting their current behavior, and their behavior was evaluated by 183 observers (total of 3,493 target self-reports, 2,973 of which had a corresponding observer report from at least one observer). Target–observer and observer–observer agreement was significant for all personality states (and was substantial for extraversion, conscientiousness, and openness to experience), and was observed in different situations, across all situations, and after accounting for normative agreement. The findings from this study—the first to examine within-person agreement on in-person behavioral states—provide evidence that people can accurately report their current behavior, that people agree on changes in behaviors across situations, and by extension that intensive assessment methodologies (such as experience-sampling methodology) have validity as assessments of momentary behavior.

Within-Person Personality Variability in the Work Context: A Blessing or a Curse for Job Performance?

“…Note, however, that our sample of observers included two different types of observers (i.e., supervisors and students), and analyses across these different rater sources (i.e., supervisor-rated personality variability predicting student-rated performance) yielded various significant effects (see Table 3), suggesting that effects may not be solely due to common rater biases (Podsakoff et al, 2003). Furthermore, recent experience sampling research has demonstrated significant agreement among different observers regarding others' personality states (Jayawickreme et al, 2023;Leikas & Lönnqvist, 2022), suggesting that other-ratings of personality states do not merely capture raters' own idiosyncratic perceptions but rather reflect actual observable behaviors. Although our overall conclusions were strengthened by a set of additional analyses that showed mostly similar results when we accounted for curvilinear effects of mean-level personality (cf., Jones et al, 2017), the main absence of cross-method effects does indicate a lack of robustness of the current findings.…”

Within-person personality variability in the work context: A blessing or a curse for job performance?

“…Note, however, that our sample of observers included two different types of observers (i.e., supervisors and students), and analyses across these different rater sources (i.e., supervisor-rated personality variability predicting student-rated performance) yielded various significant effects (see Table 3), suggesting that effects may not be solely due to common rater biases (Podsakoff et al, 2003). Furthermore, recent experience sampling research has demonstrated significant agreement among different observers regarding others’ personality states (Jayawickreme et al, 2023; Leikas & Lönnqvist, 2022), suggesting that other-ratings of personality states do not merely capture raters’ own idiosyncratic perceptions but rather reflect actual observable behaviors. Although our overall conclusions were strengthened by a set of additional analyses that showed mostly similar results when we accounted for curvilinear effects of mean-level personality (cf.…”