Transfer Learning with Convolutional Neural Networks for Rainfall Detection in Single Images

Abstract

Near real-time rainfall monitoring at local scale is essential for urban flood risk mitigation. Previous research on precipitation visual effects supports the idea of vision-based rain sensors, but tends to be device-specific. We aimed to use different available photographing devices to develop a dense network of low-cost sensors. Using Transfer Learning with a Convolutional Neural Network, the rainfall detection was performed on single images taken in heterogeneous conditions by static or moving cameras without adjusted parameters. The chosen images encompass unconstrained verisimilar settings of the sources: Image2Weather dataset, dash-cams in the Tokyo Metropolitan area and experiments in the NIED Large-scale Rainfall Simulator. The model reached a test accuracy of 85.28% and an F1 score of 0.86. The applicability to real-world scenarios was proven with the experimentation with a pre-existing surveillance camera in Matera (Italy), obtaining an accuracy of 85.13% and an F1 score of 0.85. This model can be easily integrated into warning systems to automatically monitor the onset and end of rain-related events, exploiting pre-existing devices with a parsimonious use of economic and computational resources. The limitation is intrinsic to the outputs (detection without measurement). Future work concerns the development of a CNN based on the proposed methodology to quantify the precipitation intensity.

Artificial Intelligence for Earth System Predictability (AI4ESP) (2021 Workshop Report)

“…Current state of knowledge shows that the inherent size of spatial resolution and temporal context make it imperative to have compressed representations of climate data (Kadow, Hall, and Ulbrich 2020). The compressed representation models need the ability to adapt quickly to new domains with different spatial or temporal context without significant retraining, which is also critical for sharing the knowledge (Notarangelo et al 2021). Transfer learning methods have shown promising results for filling observational gaps in climate model data (Hu, Zhang, and Zhou 2016;Notarangelo et al 2021).…”

“…The compressed representation models need the ability to adapt quickly to new domains with different spatial or temporal context without significant retraining, which is also critical for sharing the knowledge (Notarangelo et al 2021). Transfer learning methods have shown promising results for filling observational gaps in climate model data (Hu, Zhang, and Zhou 2016;Notarangelo et al 2021). Recently developed self-supervised learning (SSL) methods (Devlin et al 2018) have shown the ability to capture underlying structure in data by masking part of the data and using the remainder for predicting the masked sample.…”

A review of video‐based rainfall measurement methods

“…Tripathi and Mukhopadhyay (2012), Wahab et al (2013), Hong Wang et al (2021), and Yang et al (2021) reviewed the existing raindrop identification and removal algorithms. Whereas there is extensive literature on techniques to eliminate the visual impact of raindrops in a video or photo, there is a comparatively lower amount of research on measuring rainfall for hydrological purposes (Notarangelo et al, 2021). In contrast to vision‐based raindrop removal methods, it is challenging to measure rainfall based on vision for two aspects: accurate raindrop identification and raindrop size calculation.…”

Degradation of Commercially Available Digital Camera Images due to Variation of Rainfall Intensity in Outdoor Conditions

“…Currently, rainfall data are mainly obtained from ground observation such as rain gauge and remote sensing such as weather radar and satellites. However, rainfall data is often limited in terms of spatio-temporal resolution due to the sparseness of the ground observation networks used for both direct measurement and indirect measurement calibrations (Notarangelo et al, 2021). In addition, due mainly to the high cost of observation, a high-resolution, ground-level rainfall monitoring network has not yet been developed (Jiang et al, 2019).…”

“…As an initiative to overcome the issues mentioned above, techniques have been proposed to build sensors using low-cost equipment not used for its intended use and to combine a variety of not fully utilized technologies to make opportunistic observation (Tauro et al, 2018). For these techniques, an approach has been adopted in the form of aggregating data obtained from a high-density network built using a large number of low-cost sensors that are less accurate (Notarangelo et al, 2021). While such an approach is not as accurate as conventional rain gauges in most cases, it could provide valuable additional information when combined with conventional techniques (Tauro et al, 2018).…”