The Effectiveness of a Probabilistic Principal Component Analysis Model and Expectation Maximisation Algorithm in Treating Missing Daily Rainfall Data

Chuan, Zun Liang; Deni, Sayang Mohd; Fam, Soo-Fen; Ismail, Noriszura

doi:10.1007/s13143-019-00135-8

Cited by 10 publications

(9 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Kuantan river basin is about 440 km long and irrigates 29,300 km 2 of the area, and it is located in the eastern part of Peninsular Malaysia. This river is in the Kuantan district that covers 1,630 km² of the catchment area starting from the forest reserved area in Mukim Ulu Kuantan; it passes through the agricultural area and the state capital of Pahang before it is discharged into the South China Sea (Saeed et al, 2016;Chuan et al, 2020). Moreover, this river is the main principle tributary, which irrigates the major rural, urban, agricultural, and industrial areas of the Kuantan district.…”

Section: Study Areasmentioning

confidence: 99%

“…Data screening is a preliminary and essential procedure in inspecting and treating errors in data set, such as missing data. In practice, the missing data in the historical monthly rainfall time series data is invertible due to the meteorological extremes, recorded error, and malfunctions of instruments (Chuan et al, 2020;Chuan et al, 2018a;Chuan et al, 2018b;Chuan et al, 2018c;Saeed et al, 2016). X incomp is a matrix of size I J × of the historical monthly rainfall amount as in Equation 1:…”

Section: Data Screeningmentioning

confidence: 99%

See 1 more Smart Citation

A Comparative Effectiveness of Hierarchical and Non-hierarchical Regionalisation Algorithms in Regionalising the Homogeneous Rainfall Regions

Chuan¹,

Yusoff²,

Senawi³

et al. 2022

JST

Self Cite

View full text Add to dashboard Cite

Descriptive data mining has been widely applied in hydrology as the regionalisation algorithms to identify the statistically homogeneous rainfall regions. However, previous studies employed regionalisation algorithms, namely agglomerative hierarchical and non-hierarchical regionalisation algorithms requiring post-processing techniques to validate and interpret the analysis results. The main objective of this study is to investigate the effectiveness of the automated agglomerative hierarchical and non-hierarchical regionalisation algorithms in identifying the homogeneous rainfall regions based on a new statistically significant difference regionalised feature set. To pursue this objective, this study collected 20 historical monthly rainfall time-series data from the rain gauge stations located in the Kuantan district. In practice, these 20 rain gauge stations can be categorised into two statistically homogeneous rainfall regions, namely distinct spatial and temporal variability in the rainfall amounts. The results of the analysis show that Forgy K-means non-hierarchical (FKNH), Hartigan- Wong K-means non-hierarchical (HKNH), and Lloyd K-means non-hierarchical (LKNH) regionalisation algorithms are superior to other automated agglomerative hierarchical and non-hierarchical regionalisation algorithms. Furthermore, FKNH, HKNH, and LKNH yielded the highest regionalisation accuracy compared to other automated agglomerative hierarchical and non-hierarchical regionalisation algorithms. Based on the regionalisation results yielded in this study, the reliability and accuracy that assessed the risk of extreme hydro-meteorological events for the Kuantan district can be improved. In particular, the regional quantile estimates can provide a more accurate estimation compared to at-site quantile estimates using an appropriate statistical distribution.

show abstract

Section: Study Areasmentioning

confidence: 99%

Section: Data Screeningmentioning

confidence: 99%

A Comparative Effectiveness of Hierarchical and Non-hierarchical Regionalisation Algorithms in Regionalising the Homogeneous Rainfall Regions

Chuan¹,

Yusoff²,

Senawi³

et al. 2022

JST

Self Cite

View full text Add to dashboard Cite

show abstract

“…The missing series were completed using expected maximization approach (McLachlan and Krishnan 2007). The expected maximization approach was selected as the approach is widely used to recent studies for filling the missing time series (Ahmed et al 2017a;Alamgir et al 2015;Chuan et al 2019;Jahan et al 2019).…”

Section: Study Area and Datamentioning

confidence: 99%

Inhomogeneity detection in the precipitation series: case of arid province of Pakistan

Ahmed

Nawaz

Khan

et al. 2020

Environ Dev Sustain

View full text Add to dashboard Cite

Reliable precipitation data are often required for conducting hydro-climatological assessments. Therefore, the study aims to detect the inhomogeneity in each calendar month, mean annual and monthly time series precipitation data of Balochistan, an arid province of Pakistan. The inhomogeneity was assessed using standard normal homogeneity test, Buishand range test, Pettitt test and von Neumann ratio test at 95% confidence level. The tests were applied over the precipitation data obtained from 14 meteorological stations for the period 1961-2009. The results of the different tests were classified as 'useful', 'doubtful' and 'suspect' based on the null hypothesis. The result of calendar months and annual series indicates that most of the data were 'useful'. The output of monthly time series precipitation indicates that the data at Jiwani, Lasbela, Nokkunddi, Ormara, Pasni, Turbat and Zhob stations were 'useful' and others are classified into 'doubtful' and 'suspect' class. It was also observed that von Neumann ratio test is sensitive to minor changes in the precipitation data, and standard normal homogeneity test was found less sensitive to changes.

show abstract

“…From the reviewed literature, it can be concluded that most of the missing values in hydrological data are categorised under MAR mechanism because the presence of missing data only depends on the observed data rather than the other variables (e.g. the latitude, the longitude, or the elevation of the rainfall stations) (Chuan et al, 2019). Since several missing data estimation methods are to be used in this study, including multiple imputation approach, the missing data mechanism can be assumed to be missing at random (MAR).…”

Section: Introductionmentioning

confidence: 99%

Controlled Sampling Approach in Improving Multiple Imputation for Missing Seasonal Rainfall Data

Burhanuddin¹,

Deni²,

Shaadan³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Missingness in rainfall data is one of the well-known and challenging issues faced by meteorologists and researchers from all over the world. The problem would affect the quality of the data which is very important in representing the actual meteorological characteristics of a particular location. Therefore, the missing data should be properly treated in order to provide good quality dataset for the public domain. In furtherance of ensuring the accuracy of imputed missing data, the original structure of the rainfall data series must be specifically preserved when the data are having seasonal patterns. Most of the environmental datasets are generally characterized by outliers and seasonal patterns. These characteristics have certainly affected the performance of missing data imputation methods. The problem of missing data can be treated, but a specific structured approach must be employed when involving dataset that contains outliers and seasonal patterns. This study has highlighted and discussed the structured and comprehensive procedures on how to tackle the problem of missing data by emphasizing on controlled sampling approach for their implementation. The missing values were estimated by using multiple imputation based on block bootstrap approach associated with normal ratio methods compared to the conventional sampling (i.e. general bootstrap approach). The analysis and experimentation are illustrated using several datasets obtained for several locations in Peninsular Malaysia. The block bootstrap approach has revealed its advantage of preserving time series structure in its process and successfully improved the estimates of missing rainfall data imputation.

show abstract

The Effectiveness of a Probabilistic Principal Component Analysis Model and Expectation Maximisation Algorithm in Treating Missing Daily Rainfall Data

Cited by 10 publications

References 15 publications

A Comparative Effectiveness of Hierarchical and Non-hierarchical Regionalisation Algorithms in Regionalising the Homogeneous Rainfall Regions

A Comparative Effectiveness of Hierarchical and Non-hierarchical Regionalisation Algorithms in Regionalising the Homogeneous Rainfall Regions

Inhomogeneity detection in the precipitation series: case of arid province of Pakistan

Controlled Sampling Approach in Improving Multiple Imputation for Missing Seasonal Rainfall Data

Contact Info

Product

Resources

About