Rain event detection in commercial microwave link attenuation data using convolutional neural networks

Abstract

Abstract. Quantitative precipitation estimation with commercial microwave links (CMLs) is a technique developed to supplement weather radar and rain gauge observations. It is exploiting the relation between the attenuation of CML signal levels and the integrated rain rate along a CML path. The opportunistic nature of this method requires a sophisticated data processing using robust methods. In this study we focus on the processing step of rain event detection in the signal level time series of the CMLs, which we treat as a binary classification problem. We analyze the performance of a convolutional neural network (CNN), which is trained to detect rainfall specific attenuation patterns in CML signal levels, using data from 3904 CMLs in Germany. The CNN consists of a feature extraction and a classification part with, in total, 20 layers of neurons and 1.4 x 105 trainable parameters. With a structure, inspired by the visual cortex of mammals, CNNs use local connections of neurons to recognize patterns independent of their location in the time-series. We test the CNNs ability to generalize to CMLs and time periods outside the training data. Our CNN is trained on four months of data from 400 randomly selected CMLs and validated on two different months of data, once for all CMLs and once for the 3504 CMLs not included in the training. No CMLs are excluded from the analysis. As a reference data set we use the gauge adjusted radar product RADOLAN-RW provided by the German meteorological service (DWD). The model predictions and the reference data are compared on an hourly basis. Model performance is compared to a reference method, which uses the rolling standard deviation of the CML signal level time series as a detection criteria. Our results show that within the analyzed period of April to September 2018, the CNN generalizes well to the validation CMLs and time periods. A receiver operating characteristic (ROC) analysis shows that the CNN is outperforming the reference method, detecting on average 87 % of all rainy and 91 % of all non-rainy periods. In conclusion, we find that CNNs are a robust and promising tool to detect rainfall induced attenuation patterns in CML signal levels from a large CML data set covering entire Germany.

Rainfall estimation from a German-wide commercial microwave link network: optimized processing and validation for 1 year of data

“…"Similar" in this context means considering the size and temporal aggregation of the CML data set as well as the use of radar data as a reference for path-averaged (link-based) rain rates from CMLs. The performance measures from our results with the respective thresholds are in the same range as the performance measures from de Vos et al (2019) and Rios Gaona et al (2015). Nevertheless, the results should not be compared in a purely quantitative way, because both use different sampling strategies and span different time periods.…”

“…This, combined with the drawback of kriging that the required computation time is significantly increased (approximately 10 to 100 times slower than IDW, depending on factors such as the number of neighboring points used by a moving kriging window), meant that we decided to keep using the simple -yet robust and fast -IDW interpolation. Furthermore, it is important to note that the errors in rain rate estimation for each CML contribute most to the uncertainty of CML-derived rainfall maps (Rios Gaona et al, 2015). Hence, within the scope of this work, we focused on improving the rainfall estimation at the individual CMLs.…”

Rainfall intensity in geomorphology: Challenges and opportunities

“…Networks of disdrometers have also been deployed for areal rain studies (Jameson and Larsen, 2016), but these involve very considerable expense. New methods for recording rainfall intensity are emerging continually, including the collection of data from car windscreen wipers (Bartos et al, 2019), the use of vision from security cameras (Jiang et al, 2019), and from signal attenuation among networks of telephone towers (Polz et al, 2020).…”

Deep Learning for an Improved Prediction of Rainfall Retrievals From Commercial Microwave Links

“…Secondly, this disdrometer-trained model replaces only one of the steps among numerous successive steps in the rainfall retrieval, such as dry/wet classification, baseline estimation, and wet antenna attenuation. Such data-driven deep learning approaches for dry/wet classification have also been recently explored both by Polz et al (2019) (under review at HESSD) and Habi and Messer (2018). Similarly, one could think of a similar approach for baseline estimation and wet antenna attenuation.…”

“…Closer to the application presented here, Mishra et al (2018) have implemented a deep learning framework to distinguish dry and wet periods from communication satellite data to improve rainfall retrievals. Recently, Habi and Messer (2018) and Polz et al (2019) have also used machine learning techniques for wet‐dry classification of commercial microwave links. Similarly, there have been a few other studies using deep learning for rainfall runoff modeling (Hu et al, 2018; Kratzert et al, 2018), but according to an extensive search this is the first work that employs a recurrent neural network for improving the rainfall estimation from commercial microwave link data.…”