Characterisation of urban environment and activity across space and time using street images and deep learning in Accra

Abstract

The urban environment influences human health, safety and wellbeing. Cities in Africa are growing faster than other regions but have limited data to guide urban planning and policies. Our aim was to use smart sensing and analytics to characterise the spatial patterns and temporal dynamics of features of the urban environment relevant for health, liveability, safety and sustainability. We collected a novel dataset of 2.1 million time-lapsed day and night images at 145 representative locations throughout the Metropolis of Accra, Ghana. We manually labelled a subset of 1,250 images for 20 contextually relevant objects and used transfer learning with data augmentation to retrain a convolutional neural network to detect them in the remaining images. We identified 23.5 million instances of these objects including 9.66 million instances of persons (41% of all objects), followed by cars (4.19 million, 18%), umbrellas (3.00 million, 13%), and informally operated minibuses known as tro tros (2.94 million, 13%). People, large vehicles and market-related objects were most common in the commercial core and densely populated informal neighbourhoods, while refuse and animals were most observed in the peripheries. The daily variability of objects was smallest in densely populated settlements and largest in the commercial centre. Our novel data and methodology shows that smart sensing and analytics can inform planning and policy decisions for making cities more liveable, equitable, sustainable and healthy.

Beyond here and now: Evaluating pollution estimation across space and time from street view images with deep learning

“…All object categories are those which may vary over time at a given place, since although other static features, such as buildings or trees, may also affect noise and air pollution, their unchanging presence over daily timescales is less informative for predicting temporal variation in pollution at a single location (such as those models developed in 1a and 1b). The accuracy with which these objects could be detected in our images is given in Appendix Table C and described in previous work ( Nathvani et al, 2022 ). In this analysis, we did not use counts of cookstove, loudspeakers, market vendors or buses, due to their sparse presence in our data (<10 counts of each in 2.1 million images).…”

“…Other approaches to feature extraction from images, such as semantic segmentation, could also have been employed to provide model inputs for pollution estimation, as used in a North American study ( Qi and Hankey, 2021 ). We used objects in our second approach since the data needed to train a model, namely objects, were less resource intensive to generate within our bespoke dataset with bounding boxes ( Nathvani et al, 2022 ) as compared with pixel-level annotation, which may also be explored in future work. The object counts were obtained from training an object detection CNN, described in detail in previous work ( Nathvani et al, 2022 ), for object categories relevant to the local environmental context: persons, market vendor (a person carrying a container over their heads which is a common scene in African markets), car, taxi, pick-up truck, bus, lorry, van, tro-tro (mini buses used for public transportation), motorcycle, bicycle, market stall, loudspeaker, umbrella (commonly used to protect market and roadside vendors from the sun and rain), cookstove, cooking pot/bowl (which frequently contain wares for sale in the marketplace), food, trash, (piece of) debris, and animal.…”

“…We used objects in our second approach since the data needed to train a model, namely objects, were less resource intensive to generate within our bespoke dataset with bounding boxes ( Nathvani et al, 2022 ) as compared with pixel-level annotation, which may also be explored in future work. The object counts were obtained from training an object detection CNN, described in detail in previous work ( Nathvani et al, 2022 ), for object categories relevant to the local environmental context: persons, market vendor (a person carrying a container over their heads which is a common scene in African markets), car, taxi, pick-up truck, bus, lorry, van, tro-tro (mini buses used for public transportation), motorcycle, bicycle, market stall, loudspeaker, umbrella (commonly used to protect market and roadside vendors from the sun and rain), cookstove, cooking pot/bowl (which frequently contain wares for sale in the marketplace), food, trash, (piece of) debris, and animal. All object categories are those which may vary over time at a given place, since although other static features, such as buildings or trees, may also affect noise and air pollution, their unchanging presence over daily timescales is less informative for predicting temporal variation in pollution at a single location (such as those models developed in 1a and 1b).…”

“…We used gradient boosting machines or GBMs as the algorithm for classifying noise and PM 2.5 from specific, interpretable features, which were counts of objects detected within each image from a separate CNN ( Nathvani et al, 2022 ). GBMs are ensemble tree-based models which use “boosting”, i.e.…”

“…For GBM models in 1a, we quantified the importance of each object for prediction via its permutation importance, which calculates the reduction in the model’s accuracy on the test set, before and after randomly shuffling the values of an input feature (in our case, counts in a given object category) across images. Object counts in our data were correlated among different object categories across images ( Nathvani et al, 2022 ), e.g., people and cars. Therefore a given object’s importance score might be lower when multiple objects are used for prediction than when single objects are used, because other correlated objects capture some of the same information.…”