Using Convolutional Neural Networks to Derive Neighborhood Built Environments from Google Street View Images and Examine Their Associations with Health Outcomes

Abstract

Built environment neighborhood characteristics are difficult to measure and assess on a large scale. Consequently, there is a lack of sufficient data that can help us investigate neighborhood characteristics as structural determinants of health on a national level. The objective of this study is to utilize publicly available Google Street View images as a data source for characterizing built environments and to examine the influence of built environments on chronic diseases and health behaviors in the United States. Data were collected by processing 164 million Google Street View images from November 2019 across the United States. Convolutional Neural Networks, a class of multi-layer deep neural networks, were used to extract features of the built environment. Validation analyses found accuracies of 82% or higher across neighborhood characteristics. In regression analyses controlling for census tract sociodemographics, we find that single-lane roads (an indicator of lower urban development) were linked with chronic conditions and worse mental health. Walkability and urbanicity indicators such as crosswalks, sidewalks, and two or more cars were associated with better health, including reduction in depression, obesity, high blood pressure, and high cholesterol. Street signs and streetlights were also found to be associated with decreased chronic conditions. Chain link fence (physical disorder indicator) was generally associated with poorer mental health. Living in neighborhoods with a built environment that supports social interaction and physical activity can lead to positive health outcomes. Computer vision models using manually annotated Google Street View images as a training dataset were able to accurately identify neighborhood built environment characteristics. These methods increases the feasibility, scale, and efficiency of neighborhood studies on health.

Using Computer Vision to Annotate Video-Recoded Direct Observation of Physical Behavior

“…Adams et al found that a deep learning approach applied to Google Street View was able to classify microscale features of pedestrian streetscapes (e.g., sidewalks, walk signals) with >84% accuracy compared to human annotation [25]. Yue et al [49] also applied convolutional neural networks to Google Street View and found validation accuracy of >82% for environmental features and subsequently linked those predicted built environment features with chronic conditions and mental health [48]. Given the diminishing cost of image/video data collection and storage, image-based analyses may provide important contextual information for obesity-related research, including nutrition and physical activity [48].…”

Association of Neighborhood Racial and Ethnic Composition and Historical Redlining With Built Environment Indicators Derived From Street View Images in the US

“…We used data described by Nguyen and colleagues . The data consisted of 164 million images extracted from Google Street View’s Application Programming Interface from November 1 to 30, 2019.…”

“…We used data described by Nguyen and colleagues. 23,27,29 The data consisted of 164 million images extracted from Google Street View's Application Programming Interface from November 1 to 30, 2019. Convolutional neural networks-the state-of-art model for computer vision tasks-were used to identify objects in the collected images.…”

“…In contrast, neighborhood indicators extracted from satellite and online street-level images have been shown to be useful for studying health and socioeconomic outcomes. 23,27,28 Previous studies 23,27,29 have used street-level image data to examine the association between built environment characteristics and health but have not focused on racial and ethnic inequities. In this cross-sectional study, we used built environment indicators derived from millions of street-level images to (1) quantify racial and ethnic disparities in built environment resources and (2) quantify how the built environment mediates the association between neighborhood racial composition and health outcomes.…”

Integrating Street Views, Satellite Imageries and Remote Sensing Data Into Economics and the Social Sciences

“…Street view applications make it efficient to address various issues related to on-site crime. Deng et al (2022), He et al (2017), Jing et al (2021), Khorshidi et al (2021), Kondo et al (2017), Naik et al (2014), Sas et al (2022), Sytsma et al (2021), Zhou et al (2021) and Yue, Antonietti, et al (2022) utilised street view images to discern the characteristics of the built environment for drug activities and street robberies. These studies empirically support the broken window theory and reveal that built environments induce crime and violence.…”