Promise Into Practice: Application of Computer Vision in Empirical Research on Social Distancing

Abstract

Social scientists increasingly use video data, but large-scale analysis of its content is often constrained by scarce manual coding resources. Upscaling may be possible with the application of automated coding procedures, which are being developed in the field of computer vision. Here, we introduce computer vision to social scientists, review the state-of-the-art in relevant subfields, and provide a working example of how computer vision can be applied in empirical sociological work. Our application involves defining a ground truth by human coders, developing an algorithm for automated coding, testing the performance of the algorithm against the ground truth, and running the algorithm on a large-scale dataset of CCTV images. The working example concerns monitoring social distancing behavior in public space over more than a year of the COVID-19 pandemic. Finally, we discuss prospects for the use of computer vision in empirical social science research and address technical and ethical challenges.

Human Observers Are Accurate in Judging Personal Relationships in Real-life Settings: A Methodological Tool for Human Observational Research

“…However, if this homophily tendency is weak (e.g., because it does not manifest itself in public settings), the person similarities only offer a noisy tie-sign signal, as it may have been the case with respect to the non-significant age and ethnicity measures. This argument supports the current practice that computer vision algorithms for recognizing relationship ties focus on behaviorally displayed tie-signs (Bernasco et al 2022; Ge et al 2012).…”

An ethogram method for the analysis of human distress in the aftermath of public conflicts

“…The application of the ethogram to recognize the cues of distress is not limited to a human user. Computer vision systems are increasingly implemented to detect human behaviours; therefore, a further application might consist in developing algorithms to recognize the cues of distress in order to identify emergency situations through surveillance cameras in public spaces (Bernasco et al, 2021). Such automated coding procedure has the potential to run a large-scale analysis of the video content which until now has been constrained by manual coding.…”

An ethogram method for the analysis of human distress-related behaviours in the aftermath of public conflicts

“…The application of the ethogram to recognize the cues of distress is not limited to a human user. Computer vision systems are increasingly implemented to detect human behaviours; therefore, a further application might consist in developing algorithms to recognize the cues of distress in order to identify emergency situations through surveillance cameras in public spaces (Bernasco et al, 2023). Such automated coding procedure has the potential to run a large-scale analysis of the video content which until now has been constrained by manual coding.…”