Image-Based Surrogates of Socio-Economic Status in Urban Neighborhoods Using Deep Multiple Instance Learning

Abstract

1) Background: Evidence-based policymaking requires data about the local population's socioeconomic status (SES) at detailed geographical level, however such information is often not available, or is too expensive to acquire. Researchers have proposed solutions to estimate SES indicators by analyzing Google Street View images, however these methods are also resource-intensive, since they require large volumes of manually labeled training data. 2) Methods: We propose a methodology for automatically computing surrogate variables of SES indicators using street images of parked cars and deep multiple-instance learning. Our approach does not require any manually created labels, apart from data already available by statistical authorities, while the entire pipeline for image acquisition, parked car detection, car classification and surrogate variable computation is fully automated. The proposed surrogate variables are then used in linear regression models to estimate the target SES indicators. 3) Results: We implement and evaluate a model based on the proposed surrogate variable at 30 municipalities of varying SES in Greece. Our model has $R^2=0.76$ and correlation coefficient 0.874 with the true unemployment rate, while it achieves mean absolute percentage error 0.089 and mean absolute error 1.87 on a held-out test set. 4) Conclusions: The proposed methodology can be used to estimate socioeconomic status indicators such as unemployment rate at the local level automatically, using images of parked cars detected via Google Street View, without the need for any manual labeling effort.

Perspectivas y retos de las técnicas de inteligencia artificial en el ámbito de las ciencias sociales y de la comunicación

“…Two Sigma: Using News to Predict Stock Movements: 5 Este reto plantea la utilización de técnicas de procesamiento de lenguaje natural para analizar streams textuales de noticias para generar señales de predictividad en instrumentos financieros. Los datos constan del histórico de valores de un amplio conjunto de instrumentos financieros (cotizaciones en bolsa) y del histórico de noticias financieras emitidas por el canal Thomson Reuters, junto con varias métricas extraídas de cada noticia (compañías de la que habla la noticia, tendencia positiva o negativa, etc.).…”

A methodology for obtaining objective measurements of population obesogenic behaviors in relation to the environment

“…Inference of statistics of interest from the analysis of publicly available data. A very promising illustration of this option is our work in [19] that attempts to predict unemployment rate at a fine resolution by applying deep learning and image processing techniques to Google street view images. An example of the method for two municipalities in Greece is given in Fig.…”

“…Unemployment rate in blocks of two areas, inside the same Greek municipality. Orange and red values indicate high, while blue and green values indicate low unemployment rates as estimated by the analysis of car images appearing on Google Street View[19].…”

Introduction to the Special Issue on Image-Based Information Retrieval from the Web

“…The fourth paper of the special issue entitled “ Image-Based Surrogates of Socio-Economic Status in Urban Neighborhoods Using Deep Multiple Instance Learning ” is proposed by Christos Diou, Pantelis Lelekas, and Anastasios Delopoulos [ 4 ] and tackles a challenging problem by proposing a methodology for automatically computing surrogate variables of socio-economic status (SES) indicators using street images of parked cars and deep multiple instance learning. In principle, evidence-based policy-making requires data about the local population’s SES at detailed geographical level; however, such information is often not available, or is too expensive to acquire.…”