There is great importance in understanding whether people perceive an environment as safe or unsafe. Perceptions are influenced by the built environment, and through better understanding, design interventions can be made to improve the feeling of safety. There is a rich body of research on this topic, yet it requires a lot of manual effort. In this work, we present an approach named Computational Systematic Social Observation (CSSO) to automate the collection and analysis process. The approach uses Google Street View and the Google Vision API to extract characteristics (herein referred to as features) of the built environment that is used to automate the process of understanding whether people will feel fear or safety. In testing this approach, we extracted $$\tilde{1}$$
1
~
.3M images for the 100 locations and identified 297 features of the built environment. A measure of dependency demonstrated that some are more strongly associated with areas where people express a feeling of safety or fear. Further, through empirical testing, it is observed that these features can be used for classification. The results demonstrate the potential of the technique and were compared with human coders. The presented methodology and exploratory research provide a foundation for systematic computational observation to identify environmental correlates of fear of crime.