Estimating missing data using novel correlation maximization based methods

Sefidian, Amir Masoud; Daneshpour, Negin

doi:10.1016/j.asoc.2020.106249

Cited by 25 publications

(9 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The importance of obtaining high quality data is not limited to the field of process mining. Various research papers on data repair (e.g., [32][33][34][35][36][37]) and survey papers (e.g., [38,39]) exist. These are not concerned with the data quality of the event log in process mining.…”

Section: Related Workmentioning

confidence: 99%

Extraction of Missing Tendency Using Decision Tree Learning in Business Process Event Log

Horita

Kurihashi

Miyamori

2020

Data

View full text Add to dashboard Cite

In recent years, process mining has been attracting attention as an effective method for improving business operations by analyzing event logs that record what is done in business processes. The event log may contain missing data due to technical or human error, and if the data are missing, the analysis results will be inadequate. Traditional methods mainly use prediction completion when there are missing values, but accurate completion is not always possible. In this paper, we propose a method for understanding the tendency of missing values in the event log using decision tree learning without supplementing the missing values. We conducted experiments using data from the incident management system and confirmed the effectiveness of our method.

show abstract

Section: Related Workmentioning

confidence: 99%

Extraction of Missing Tendency Using Decision Tree Learning in Business Process Event Log

Horita

Kurihashi

Miyamori

2020

Data

View full text Add to dashboard Cite

show abstract

“…The performance of each imputation method can be varied dependent on the types of missing data and datasets. In the imputation of the missing data, the current approaches usually use the similarities of missing rows with the other rows in the dataset and the correlations of the features (Sefidian & Daneshpour, 2020). Therefore, the missing data imputation methods can be partitioned into two main categories, which are global missing data imputation methods and local missing data imputation methods (Cheng, Law & Siu, 2012;Feng et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

Advanced methods for missing values imputation based on similarity learning

Fouad

Ismail

Azar

et al. 2021

PeerJ Computer Science

View full text Add to dashboard Cite

The real-world data analysis and processing using data mining techniques often are facing observations that contain missing values. The main challenge of mining datasets is the existence of missing values. The missing values in a dataset should be imputed using the imputation method to improve the data mining methods’ accuracy and performance. There are existing techniques that use k-nearest neighbors algorithm for imputing the missing values but determining the appropriate k value can be a challenging task. There are other existing imputation techniques that are based on hard clustering algorithms. When records are not well-separated, as in the case of missing data, hard clustering provides a poor description tool in many cases. In general, the imputation depending on similar records is more accurate than the imputation depending on the entire dataset's records. Improving the similarity among records can result in improving the imputation performance. This paper proposes two numerical missing data imputation methods. A hybrid missing data imputation method is initially proposed, called KI, that incorporates k-nearest neighbors and iterative imputation algorithms. The best set of nearest neighbors for each missing record is discovered through the records similarity by using the k-nearest neighbors algorithm (kNN). To improve the similarity, a suitable k value is estimated automatically for the kNN. The iterative imputation method is then used to impute the missing values of the incomplete records by using the global correlation structure among the selected records. An enhanced hybrid missing data imputation method is then proposed, called FCKI, which is an extension of KI. It integrates fuzzy c-means, k-nearest neighbors, and iterative imputation algorithms to impute the missing data in a dataset. The fuzzy c-means algorithm is selected because the records can belong to multiple clusters at the same time. This can lead to further improvement for similarity. FCKI searches a cluster, instead of the whole dataset, to find the best k-nearest neighbors. It applies two levels of similarity to achieve a higher imputation accuracy. The performance of the proposed imputation techniques is assessed by using fifteen datasets with variant missing ratios for three types of missing data; MCAR, MAR, MNAR. These different missing data types are generated in this work. The datasets with different sizes are used in this paper to validate the model. Therefore, proposed imputation techniques are compared with other missing data imputation methods by means of three measures; the root mean square error (RMSE), the normalized root mean square error (NRMSE), and the mean absolute error (MAE). The results show that the proposed methods achieve better imputation accuracy and require significantly less time than other missing data imputation methods.

show abstract

“…Gözlemsel verileri analiz ederken karşılaşılan en yaygın sorun eksik verilerdir. Gerçek veri kümeleri, çeşitli nedenlerden dolayı eksik verilerden oluşabilir [1]. Endüstri, tıp, ticaret ve bilimsel araştırmalar gibi [2] çok farklı kaynaklardan alınarak oluşturulan veri setlerinde eksik bilgiler ile karşılaşılabilir [3].…”

Section: Introductionunclassified

“…Eksik veri terminolojisi ilk kez Little ve Rubin tarafından kullanılmıştır [8]. Eksik değerleri ele almak için iyi bilinen ve hesaplama açısından basit birkaç yaklaşım vardır [1]:…”

Section: Introductionunclassified

Benzetimli Tavlama Algoritması İle Eksik Veri Tamamlama

Metin¹

2021

Fırat Üniversitesi Mühendislik Bilimleri Dergisi

View full text Add to dashboard Cite

Öz: İstatiksel birçok yöntem eksik değerlere sahip veri setleri üzerinde çalışma kapasitesine sahip değildir. Bu nedenle, girdi olarak yalnızca tam veriyi kabul eden modellerin tahmin performansı önemli ölçüde düşmektedir. Eksik verilerin tamamlanması bunun için veri analizlerinde önemli bir yere sahiptir. Bu çalışmada kullanılan veri seti üzerinde eksik olan verilerin tamamlanma probleminin çözümünde sezgisel optimizasyon yöntemi olan Benzetimli Tavlama Algoritması(BTA) kullanılmıştır. Modern sezgisel teknikler, bir problem çözümünde, kendi yerel arama sistemleri ile en iyi sonuca ulaşmayı amaçlamaktadırlar. BTA performansını etkileyen en önemli değer başlangıç sıcaklık değeri (T0) olduğundan üç farklı sıcaklık değeri ile sonuçlar alınmıştır. To=100.000 değeri için %68, To=10.000 için %51 ve To=1.000 için %46'lik bir başarı elde edilmiştir.

show abstract

Estimating missing data using novel correlation maximization based methods

Cited by 25 publications

References 41 publications

Extraction of Missing Tendency Using Decision Tree Learning in Business Process Event Log

Extraction of Missing Tendency Using Decision Tree Learning in Business Process Event Log

Advanced methods for missing values imputation based on similarity learning

Benzetimli Tavlama Algoritması İle Eksik Veri Tamamlama

Contact Info

Product

Resources

About