A study on missing values imputation using K-Harmonic means algorithm: Mixed datasets

Anwar, Taufik; Siswantining, Titin; Sarwinda, Devvi; Soemartojo, Saskya Mary; Bustamam, Alhadi

doi:10.1063/1.5141651

Cited by 7 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, ML-based imputation methods have been gradually emerging: e.g., generative adversarial networks (GAN) for missing data [19], [20]. K-Harmonic mean imputation [21], sequential regression multivariate imputation [22], Fuzzy C-Means imputation [23], and predictive mean matching [24] are also noteworthy. These existing imputation methods often require expertlevel distributional assumptions that are difficult for general researchers.…”

Section: Introductionmentioning

confidence: 99%

Ultra Data-Oriented Parallel Fractional Hot-Deck Imputation With Efficient Linearized Variance Estimation

Yang

Kwon

Kim

et al. 2023

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Parallel fractional hot-deck imputation (P-FHDI [1]) is a general-purpose, assumption-free tool for handling item nonresponse in big incomplete data by combining the theory of FHDI and parallel computing. FHDI cures multivariate missing data by filling each missing unit with multiple observed values (thus, hot-deck) without resorting to distributional assumptions. P-FHDI can tackle big incomplete data with millions of instances (big-n) or 10, 000 variables (big-p). However, handling ultra incomplete data (i.e., concurrently big-n and big-p) with tremendous instances and high dimensionality has posed problems to P-FHDI due to excessive memory requirement and execution time. To tackle the aforementioned challenges, we propose the ultra data-oriented P-FHDI (named UP-FHDI) capable of curing ultra incomplete data. In addition to the parallel Jackknife method, this paper enables a computationally efficient ultra data-oriented variance estimation using parallel linearization techniques. Results confirm that UP-FHDI can tackle an ultra dataset with one million instances and 10, 000 variables. This paper illustrates the special parallel algorithms of UP-FHDI and confirms its positive impact on the subsequent deep learning performance.

show abstract

Section: Introductionmentioning

confidence: 99%

Ultra Data-Oriented Parallel Fractional Hot-Deck Imputation With Efficient Linearized Variance Estimation

Yang

Kwon

Kim

et al. 2023

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

show abstract

“…It also offers variance estimation using the Jackknife replication method. Besides, other popular imputation methods include multiple imputation with predictive mean matching (PMM) by [18], sequential regression multivariate imputation by [19], Fuzzy C-Means (FCM) imputation by [20], and K-Harmonic mean imputation by [21].…”

Section: Introductionmentioning

confidence: 99%

Parallel Fractional Hot-Deck Imputation and Variance Estimation for Big Incomplete Data Curing

Yang

Kim

Cho

2022

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

The fractional hot-deck imputation (FHDI) is a general-purpose, assumption-free imputation method for handling multivariate missing data by filling each missing item with multiple observed values without resorting to artificially created values. The corresponding R package FHDI [1] holds generality and efficiency, but it is not adequate for tackling big incomplete data due to the requirement of excessive memory and long running time. As a first step to tackle big incomplete data by leveraging the FHDI, we developed a new version of a parallel fractional hot-deck imputation (named as P-FHDI) program suitable for curing large incomplete datasets. Results show a favorable speedup when the P-FHDI is applied to big datasets with up to millions of instances or 10,000 of variables. This paper explains the detailed parallel algorithms of the P-FHDI for large instances (big-n) or high-dimensionality (big-p) datasets and confirms the favorable scalability. The proposed program inherits all the advantages of the serial FHDI and enables a parallel variance estimation, which will benefit a broad audience in science and engineering.

show abstract

“…Unlike supervised learning, clustering is considered an unsupervised learning method since there is no supplied ground truth from the data which is known as the target variable in supervised learning. Further inspection allows only data structure investigation based on data points grouping into definitive subgroups (clusters) [15].…”

Section: Clusteringmentioning

confidence: 99%

Students’ Preparedness to Learn in e-Learning Environment and their Perception on The MPKT Lecturers’ Readiness to Manage Online Class

Siswantining¹,

Al-Ash²,

Junus³

et al. 2021

Adv. sci. technol. eng. syst. j.

Self Cite

View full text Add to dashboard Cite

This study has two objectives: to determine the level of readiness of first-year undergraduate students at the Universitas Indonesia (UI) and to investigate student's perception of MPKT (Integrated Character Development course) lecturers' readiness to manage online learning class. Proportional cluster sampling was applied, and 1466 freshmen from thirteen faculties participated. Data clustering and imputation of missing values were utilized to analysis the data. Clustering based on gender, faculty, previous elearning experience ** were applied. The study shows that students perceived themselves as being ready to learn in an e-learning environment except Computer Science students who have been more exposed to e-learning and implemented online collaborative learning. Most students agree that MPKT lecturers are able to teach well except those of the Faculty of Computer Science, Faculty of Pharmacy, and Faculty of Social and Political Sciences who think that the teaching ability of lecturers need to be improved. Recommendations and future research topics are proposed based on the study results and in-depth interviews with some experienced online lecturers.

show abstract

A study on missing values imputation using K-Harmonic means algorithm: Mixed datasets

Cited by 7 publications

References 6 publications

Ultra Data-Oriented Parallel Fractional Hot-Deck Imputation With Efficient Linearized Variance Estimation

Ultra Data-Oriented Parallel Fractional Hot-Deck Imputation With Efficient Linearized Variance Estimation

Parallel Fractional Hot-Deck Imputation and Variance Estimation for Big Incomplete Data Curing

Students’ Preparedness to Learn in e-Learning Environment and their Perception on The MPKT Lecturers’ Readiness to Manage Online Class

Contact Info

Product

Resources

About