Can You Really Study an Army on the Internet? Comparing How Status Tasks Perform in the Laboratory and Online Settings

Abstract

Laboratory experiments have a long history within sociology, with their ability to test causality and their utility for directly observing behavior providing key advantages. One influential social psychological field, status characteristics and expectation states theory, has almost exclusively used laboratory experiments to test the theory. Unfortunately, laboratory experiments are resource intensive, requiring a research pool, laboratory space, and considerable amounts of time. For these and other reasons, social scientists are increasingly exploring the possibility of moving experiments from the lab to an online platform. Despite the advantages of the online setting, the transition from the lab is challenging, especially when studying behavior. In this project, we develop methods to translate the traditional status characteristics experimental setting from the laboratory to online. We conducted parallel laboratory and online behavioral experiments using three tasks from the status literature, comparing each task’s ability to differentiate on the basis of status distinctions. The tasks produce equivalent results in the online and laboratory environment; however, not all tasks are equally sensitive to status differences. Finally, we provide more general guidance on how to move vital aspects of laboratory studies, such as debriefing, suspicion checks, and scope condition checks, to the online setting.

Gender Dynamics in Human-AI Role-Taking

“…Crowdsource platforms offer greater demographic diversity than traditional university samples (Aguinis, Villamor, & Ramani, 2021) and can be utilized for theory testing (Manago, Mize, & Doan, 2021), as we have done in the present work. Although our primary analysis is a quantified test of specific research questions, we supplement these findings with qualitative…”

Assessing the Causal Link between the COVID-19 Pandemic and Racial Discrimination

“…This is especially valuable when studying racial attitudes that are likely to be influenced by respondents’ political ideology and educational background. Research suggests that online samples yield reliable, high-quality data for studies in social psychology (Anson 2018; Manago, Mize, and Doan 2021; Paolacci and Chandler 2014).…”

Facing Others’ Trauma: A Role-Taking Theory of Burnout

“…We recruited participants from Prolific Academic (http://www.prolific.co), an online crowdsource research platform. Crowdsource platforms are suitable for theory testing (Manago, Mize, and Doan 2021), as we do in the present work. Although these platforms often produce younger and more educated samples than U.S. workers in general (Hitlin 2016), they provide satisfactory data quality, with greater diversity than traditional university student samples (Hays, Liu, and Kapteyn 2015; Pe’er et al 2021).…”