Third-party rainfall observations could provide an improvement of the current official observation network for rainfall monitoring. Although third-party weather stations can provide large quantities of near-real-time rainfall observations at fine temporal and spatial resolutions, the quality of these data is susceptible due to variations in quality control applied and there is a need to provide greater confidence in them. In this study, we develop an automatic quality evaluation procedure for daily rainfall observations collected from third-party stations in near real time. Australian Gridded Climate Data (AGCD) and radar Rainfields data have been identified as two reliable data sources that can be used for assessing third-party observations in Australia. To achieve better model interpretability and scalability, these reference data sources are used to provide separate tests rather than a complex single test on a third-party data point. Based on the assumption that the error of a data source follows a Gaussian distribution after a log-sinh transformation, each test issues a p-value-based confidence score as a measure of quality and the confidence of the third-party data observation. The maximum of confidence scores from individual tests is used to merge these tests into a single result which provides overall assessment. We validate our method with synthetic datasets based on high-quality rainfall observations from 100 Bureau of Meteorology (BoM) of Australia stations across Australia and apply it to evaluate real third-party rainfall observations owned by the Department of Primary Industries and regional development (DPIRD) of Western Australia. Our method works well with the synthetic datasets and can detect 76.7% erroneous data while keeping the false alarm rate as low as 1.7%. We also discuss the possibility of using other reference datasets, such as numerical weather prediction data and satellite rainfall data.
