Correlation-based feature partition regression method for unsupervised anomaly detection

Liu, Zhiyu; Gao, Xin; Jia, Xin; Xue, Bing; Fu, Shiyuan; Li, Kangsheng; Huang, Xu; Huang, Zijian

doi:10.1007/s10489-022-03247-3

Cited by 6 publications

(4 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper uses the Pearson correlation coefficient based on the GIS platform to calculate the degree of correlation between sample features. The Pearson correlation coefficient is generally used to measure the degree of linear correlation between two vectors [42]. Based on the information provided, the dataset "land cover type" has a significant negative correlation with the "NDVI data", with a correlation coefficient of −0.566, which exceeds the threshold of 0.5 (Table 1).…”

Section: Geological Datamentioning

confidence: 99%

Collapse Susceptibility Assessment in Taihe Town Based on Convolutional Neural Network and Information Value Method

Li,

Hu,

Lin

et al. 2024

Water

View full text Add to dashboard Cite

The cause mechanism of collapse disasters is complex and there are many influencing factors. Convolutional Neural Network (CNN) has a strong feature extraction ability, which can better simulate the formation of collapse disasters and accurately predict them. Taihe town’s collapse threatens roads, buildings, and people. In this paper, road distance, water distance, normalized vegetation index, platform curvature, profile curvature, slope, slope direction, and geological data are used as input variables. This paper generates collapse susceptibility zoning maps based on the information value method (IV) and CNN, respectively. The results show that the accuracy of the susceptibility assessment of the IV method and the CNN method is 85.1% and 87.4%, and the accuracy of the susceptibility assessment based on the CNN method is higher. The research results can provide some reference for the formulation of disaster prevention and control strategies.

show abstract

Section: Geological Datamentioning

confidence: 99%

Collapse Susceptibility Assessment in Taihe Town Based on Convolutional Neural Network and Information Value Method

Li,

Hu,

Lin

et al. 2024

Water

View full text Add to dashboard Cite

show abstract

“…When selecting one of the data dimensionality reduction techniques, it is essential to consider that in methods based on data partitioning, the high density or low variance of data in features usually does not lead to meaningful dimensionality reduction. Consequently, it is better to opt for a dimensionality reduction method that selects dimensions with the highest variance and eliminates features with low variance [12,13]. One of the well-known dimensionality reduction methods is Principal Component Analysis (PCA) [14], which identifies correlations between variables.…”

Section: Data Preprocessing and Dimensionality Reductionmentioning

confidence: 99%

Extended Random Histogram Forest for Unsupervised Anomaly Detection

Ansarirad,

Hashemi

2023

Preprint

View full text Add to dashboard Cite

Anomaly detection is a fundamental task in the field of unsupervised machine learning, aimed at identifying instances that significantly deviate from other input data. This problem has various applications, including identifying defective products in industries, detecting network intrusions, medical diagnostics, and many other cases. Despite extensive research conducted in this field, a solution with satisfactory performance under all conditions and types of data has not yet been achieved. One effective unsupervised method is the Random Histogram Forest (RHF) approach, which utilizes a probabilistic approach. Based on the evaluation metric of average precision of the area under the precision-recall curve (AP), this approach has shown better performance compared to other methods in terms of AP. However, due to its inherent mechanism, this approach also has limitations. In this article, these limitations are examined, and solutions and methods are proposed to address these constraints, and an extension approach called Extended Random Histogram Forest (ERHF) is introduced for unsupervised anomaly detection. ERHF utilizes data projection and dimensionality reduction methods to enhance the compatibility between the data and the internal mechanism of RHF. Instead of randomly selecting splitting points in the process of constructing random trees, the Skewness score is used to choose more targeted splitting points. Additionally, multidimensional hyperplanes with random slopes are used for data partitioning instead of pointwise splits. Furthermore, the mechanism for calculating the anomaly score of instances is modified to enable the use of subsampling. The performance evaluation of the proposed ERHF method using various metrics (AP, AUC, Recall, F1) on ODDS datasets demonstrates that ERHF significantly outperforms the RHF method.

show abstract

“…The embedded method combines the filter and wrapper techniques and it derives the amount of significance of features from the classifier's internal parameters. Recently plenty of results have been published in different feature selection ideas such as robust hierarchical feature selection, unsupervised feature selection and multi-label feature selection, see for example [57]- [61] and references therein. In an expert system, feature selection has evolved into a critical pre-processing phase.…”

Section: Introductionmentioning

confidence: 99%

Multi-Criteria Decision Making Methods for Ensemble Feature Selection using q-Rung Orthopair Hesitant Fuzzy Distance and Similarity Measures

Kesavan

Seethapathy

Satheeshkumar

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper deals with ensemble feature selection using the q-rung orthopair hesitant fuzzy multi-criteria decision-making (MCDM) methods including VIse Kriterijumska Optimizacija I Kompromisno Resenje (VIKOR), Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) and Combinative distance-based assessment (CODAS). The novalty of this paper is to design the three MCDM algorithms based on q-rung orthopair hesitant fuzzy sets with different distance and similarity measures. The well known distance and similarity measures are to be taken such as Hausdorff measure, hybrid Hausdorff and distance measure, synergetic measure, similarity for Hausdorff measure, similarity for hybrid Hausdorff and distance measure, similarity for synergetic measure, ordered Hausdorff measure, ordered hybrid Hausdorff and distance measure, similarity for ordered Hausdorff measure and similarity for ordered hybrid Hausdorff and distance measure This is the first time in the literature, an ensemble feature selection problem is modeled as a q-rung orthopair hesitant fuzzy MCDM extended to VIKOR, TOPSIS and CODAS techniques with distance and similarity measures. By using q-ROHFS VIKOR, TOPSIS and CODAS methods, a score is assigned to each feature based on the values of the preference matrix. At last, an output rank vector is produced for all features, from which the user can select the desired number of features. To prove the efficiency and optimality of our proposed method, we compared with the basic filter-based feature selections and ensemble feature selection by using feature ranking strategy. Our method is superior and efficient than the ensemble methods based on the accuracy and F-score levels.

show abstract

Correlation-based feature partition regression method for unsupervised anomaly detection

Cited by 6 publications

References 27 publications

Collapse Susceptibility Assessment in Taihe Town Based on Convolutional Neural Network and Information Value Method

Collapse Susceptibility Assessment in Taihe Town Based on Convolutional Neural Network and Information Value Method

Extended Random Histogram Forest for Unsupervised Anomaly Detection

Multi-Criteria Decision Making Methods for Ensemble Feature Selection using q-Rung Orthopair Hesitant Fuzzy Distance and Similarity Measures

Contact Info

Product

Resources

About