Predicting Economic Development using Geolocated Wikipedia Articles

Abstract

Progress on the UN Sustainable Development Goals (SDGs) is hampered by a persistent lack of data regarding key social, environmental, and economic indicators, particularly in developing countries. For example, data on poverty -the first of seventeen SDGs -is both spatially sparse and infrequently collected in Sub-Saharan Africa due to the high cost of surveys. Here we propose a novel method for estimating socioeconomic indicators using open-source, geolocated textual information from Wikipedia articles. We demonstrate that modern NLP techniques can be used to predict community-level asset wealth and education outcomes using nearby geolocated Wikipedia articles. When paired with nightlights satellite imagery, our method outperforms all previously published benchmarks for this prediction task, indicating the potential of Wikipedia to inform both research in the social sciences and future policy decisions.

Nowcasting food insecurity on a global scale

“…Therefore, during the last few years, researchers have begun to investigate the potential of non-traditional data and new computational methods to estimate vulnerabilities and socioeconomic characteristics when primary data is not available. In these studies, mobile phone data [2], satellite imagery [3], a combination of both [4,5], geolocated Wikipedia articles [6] or Tweets [7], and social media advertising data [8], have been used in combination with state-of-theart machine learning methods to provide reliable estimates of poverty at different spatial resolutions for several Sub-Saharan African countries as well as Southern and Southeastern Asian ones.…”

Big Data and the United Nations Sustainable Development Goals (UN SDGs) at a Glance

“…Researchers can apply advanced data mining and machine learning techniques to improve their analyses of existing data. The are some examples where Big Data have contributed to improve macroeconomic forecasts and indicators-for example, Google data [128,129], economic news [125], web data [130,131] and individual bank card transaction data [132]. Others have exploited innovative methodologies or techniques Big Data has to improve practical functionality of economic modeling-for instance, early warnings of economic crisis using artificial neural networks [133] or trade nowcasting [134].…”

Using Satellite Imagery to Understand and Promote Sustainable Development

“…Approaches include stacking the inputs as additional channels of a single network, or multi-branch architectures where data modalities are processed separately to extract features which are then concatenated before a final prediction layer. Examples of this approach include models that combine multiple sources of satellite information 30 or models that combine imagery with data from weather sensors, 48 cell phones, 29 Wikipedia, 49 social media, 50 street-level imagery 51 or Open Street Map 52 to predict development-related outcomes.…”

“…Second, although small samples make generalization tenuous, studies that made predictions at more aggregate spatial scales, and studies that combined satellite information with data from other sources, tended to outperform village-level satellite-only models. These data fusion approaches have become increasingly common, with researchers demonstrating how combining imagery with data from cell phones, 29 Wikipedia, 49 social media, 50 or Open Street Map 52 can improve predictions.…”