Applying machine learning and geolocation techniques to social media data (Twitter) to develop a resource for urban planning

Abstract

With all the recent attention focused on big data, it is easy to overlook that basic vital statistics remain difficult to obtain in most of the world. What makes this frustrating is that private companies hold potentially useful data, but it is not accessible by the people who can use it to track poverty, reduce disease, or build urban infrastructure. This project set out to test whether we can transform an openly available dataset (Twitter) into a resource for urban planning and development. We test our hypothesis by creating road traffic crash location data, which is scarce in most resource-poor environments but essential for addressing the number one cause of mortality for children over five and young adults. The research project scraped 874,588 traffic related tweets in Nairobi, Kenya, applied a machine learning model to capture the occurrence of a crash, and developed an improved geoparsing algorithm to identify its location. We geolocate 32,991 crash reports in Twitter for 2012–2020 and cluster them into 22,872 unique crashes during this period. For a subset of crashes reported on Twitter, a motorcycle delivery service was dispatched in real-time to verify the crash and its location; the results show 92% accuracy. To our knowledge this is the first geolocated dataset of crashes for the city and allowed us to produce the first crash map for Nairobi. Using a spatial clustering algorithm, we are able to locate portions of the road network (<1%) where 50% of the crashes identified occurred. Even with limitations in the representativeness of the data, the results can provide urban planners with useful information that can be used to target road safety improvements where resources are limited. The work shows how twitter data might be used to create other types of essential data for urban planning in resource poor environments.

The Plebeian Algorithm: A Democratic Approach to Censorship and Moderation

“…Myriad studies have been conducted surrounding infodemics [ 51 - 53 ]. This includes the correlation between geolocation of social media connections and various social determinants (eg, race, sex, and socioeconomic status) [ 51 ], and a study [ 54 ] determining optimal methods of geolocation on social media.…”

“…This includes the correlation between geolocation of social media connections and various social determinants (eg, race, sex, and socioeconomic status) [ 51 ], and a study [ 54 ] determining optimal methods of geolocation on social media. Two additional studies discussed the sociological consequences of geolocation in the context of social media, namely, the detection and reduction of youth cannabis consumption [ 52 ] and the applications of geolocation to urban planning [ 53 ].…”

The moderating effects of entertainers on public engagement through government activities in social media during the COVID-19

“…In addition, according to the 45th China Statistical Report on Internet Development, which was published by the China Internet Network Information Center (CNNIC), at the end of March 2020, the number of active Weibo users was 70 million, which was more than half of all Chinese netizens on average each month. In past years, Weibo has been widely used by many studies ( Jain and Katarya, 2019 , Han and Wang, 2019 , Milusheva et al, 2021 , Wang et al, 2018 , Shan et al, 2021 ). Because the aim of this study was to investigate the moderating effects of entertainers on official information diffusion on social media, only the original posts were considered.…”

A data fusion approach to the estimation of temporary populations: An application to Australia

“…To better protect individual privacy, Twitter removed the service that automatically tags precise locations in June 2019. Although missing coordinates can be inferred as for example in [ 43 ], the policies around the usage of social media data can change over time posing significant challenges for consistent and reproducible research.…”