Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States

Gebru, Timnit; Krause, Jonathan; Wang, Yilun; Chen, Duyun; Deng, Jia; Aiden, E; Li, Feifei

doi:10.1073/pnas.1700035114

Cited by 399 publications

(262 citation statements)

References 20 publications

Supporting

Mentioning

258

Contrasting

Unclassified

Order By: Relevance

“…Early work used low-level image features or mined visually distinctive patches [9,29,8] to predict geo-spatial properties such as perceived safety of cities [2,25,26], or ecological properties such as snow or cloud cover [41,34,24]. Advances in visual recognition has enabled more sophisticated analysis, such as the analysis of demographics by recognizing the make and model of cars in Street View [10]. However, while this work is exciting, the focus has been on using vision to predict known geo-spatial trends rather than discover new ones.…”

Section: Related Workmentioning

confidence: 99%

GeoStyle: Discovering Fashion Trends and Events

Mall

Matzen

Hariharan

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Understanding fashion styles and trends is of great potential interest to retailers and consumers alike. The photos people upload to social media are a historical and public data source of how people dress across the world and at different times. While we now have tools to automatically recognize the clothing and style attributes of what people are wearing in these photographs, we lack the ability to analyze spatial and temporal trends in these attributes or make predictions about the future. In this paper we address this need by providing an automatic framework that analyzes large corpora of street imagery to (a) discover and forecast longterm trends of various fashion attributes as well as automatically discovered styles, and (b) identify spatio-temporally localized events that affect what people wear. We show that our framework makes long term trend forecasts that are > 20% more accurate than prior art, and identifies hundreds of socially meaningful events that impact fashion across the globe. The supplementary material can be found at

show abstract

Section: Related Workmentioning

confidence: 99%

GeoStyle: Discovering Fashion Trends and Events

Mall

Matzen

Hariharan

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…We have presented a simple but novel unsupervised approach to extract and interrogate visual latent responses from urban images. This research sits in contrast to previous works which focused on supervised learning [9,18,23,27] and unsupervised learning for reconstruction [14,31]. Through visualisation, both geographically and generative, and prediction experiments we were able to retrieve meaning from these latent responses.…”

Section: Discussionmentioning

confidence: 74%

“…One such example is StreetScore where [23] collected human perception data from street images through a crowd-sourced survey (Place Pulse 2.0) which are then used to predict the perceived safety of a place [8]. Another example is the work of Gebru et al [9] whom extracted features such as car types from Google StreetView images to predict the income, race, education, and voting patterns for cities in the States. We have also seen the use of urban images [27] to predict scenicness ratings which were found to affect urban wellbeing.…”

Section: Related Work 21 Streetviewsmentioning

confidence: 99%

An unsupervised approach to geographical knowledge discovery using street level and street network images

Law

Neira

2019

Proceedings of the 3rd ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery

View full text Add to dashboard Cite

Recent researches have shown the increasing use of machine learning methods in geography and urban analytics, primarily to extract features and patterns from spatial and temporal data. Research, integrating geographical processes in machine learning models and, leveraging on geographical information to better interpret these methods had been sparse. This research contributes to the ladder, where we show how latent variables learned from unsupervised learning methods can be used for geographic knowledge discovery. In particular, we propose a simple and novel approach called Convolutional-PCA (ConvPCA) which are applied on both street level and street network images in finding a set of uncorrelated visual latent responses. The approach allows for meaningful explanations using a combination of, geographical and generative visualizations to explore the latent space, and to show how the learned embeddings can be used to predict urban characteristics such as street-level enclosures and street network density.

show abstract

“…Using street view imagery researchers showed how it can be a predictor to several urban socio-economic measures in cities. Gebru et al developed an object detection model to extract cars from street view images, the paper then proceed by doing an image classification of the make, model and age of cars seen in a neighborhood area [8]. Then a lookup for the prices of the cars seen in images is performed against a database of their expected prices.…”

Section: Literaturementioning

confidence: 99%

Streetify: Using Street View Imagery And Deep Learning For Urban Streets Development

Alhasoun

González

2019

2019 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

The classification of streets on road networks has been focused on the vehicular transportational features of streets such as arterials, major roads, minor roads and so forth based on their transportational use. City authorities on the other hand have been shifting to more urban inclusive planning of streets, encompassing the side use of a street combined with the transportational features of a street. In such classification schemes, streets are labeled for example as commercial throughway, residential neighborhood, park etc. This modern approach to urban planning has been adopted by major cities such as the city of San Francisco, the states of Florida and Pennsylvania among many others. Currently, the process of labeling streets according to their contexts is manual and hence is tedious and time consuming. In this paper, we propose an approach to collect and label imagery data then deploy advancements in computer vision towards modern urban planning. We collect and label street imagery then train deep convolutional neural networks (CNN) to perform the classification of street context. We show that CNN models can perform well achieving accuracies in the 81% to 87%, we then visualize samples from the embedding space of streets using the t-SNE method and apply class activation mapping methods to interpret the features in street imagery contributing to output classification from a model.

show abstract

Using deep learning and Google Street View to estimate the demographic makeup of neighborhoods across the United States

Cited by 399 publications

References 20 publications

GeoStyle: Discovering Fashion Trends and Events

GeoStyle: Discovering Fashion Trends and Events

An unsupervised approach to geographical knowledge discovery using street level and street network images

Streetify: Using Street View Imagery And Deep Learning For Urban Streets Development

Contact Info

Product

Resources

About