Citizen Social Scientists’ Observations on Complex Tasks Match Trained Research Assistants’, Suggesting Lived Experiences are Valuable in Data Collection

Abstract

Can citizen scientists reliably and meaningfully observe and record complex social phenomena? To explore this question in more detail, we recruited 162 diverse citizen scientists to identify publicly located structural markers of that exclude the elderly, women, large-bodied people, and non-white people in Phoenix, Arizona. The quality of observations by citizen scientists was assessed against data collected on the same observational task by (1) three professional social scientists with expertise on discrimination and (2) 33 trained research assistants. We found that the performance of citizen social scientists is similar to that of trained research assistants, even while both performed very differently from professional social scientists. The main finding is that citizen social scientists who had self-reported with specific social categories relevant to the social exclusions were roughly equal observers to those who did not identify as belonging to those categories, an unexpected finding. Likewise, citizen social scientists who reported experiences of discrimination were not more likely to observe more social exclusions or discrimination in public places than those who did not report such experiences. One key implication is that detailed input from volunteers in how they approach social science tasks could illuminate how social categories (like age or gender) matter for recruitment and performance in citizen social science research.