Robust geodesic based outlier detection for class imbalance problem

Shi, Canghong; Li, Xiaojie; Lv, Jiancheng; Yin, Jinghua; Mumtaz, Imran

doi:10.1016/j.patrec.2020.01.028

Cited by 6 publications

(6 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In most data collection processes, due to several causes such as variations in measurement methods, human negligence, or experimental errors, some samples are extremely different from the rest of the samples collected and they are referred to as outliers (Chen, Miao, and Zhang 2010;Li, Lv, and Yi 2016). Similar to noises, the outliers may affect the learning performance of the model and even make it difficult to interpret the analysis results (Shi et al 2020). So, it is very important to identify and eliminate outliers in data and thus several useful approaches have been developed to handling outliers to improve the learning performance.…”

Section: Anomaly Detection Methodsmentioning

confidence: 99%

“…Here, if OBN x i ð Þ for a sample x i is greater than the average OBN value of all the samples, the sample x i is classified as an outlier (Gupta, Bhattacharjee, and Bishnu 2019). Shi et al (2020) proposed a geodesic-based outlier detection algorithm that considers both the global disconnection score and the local realness which can evaluate the degree of outlier of each sample and connectivity between samples as the detection measure of outliers. In particular, they constructed a global disconnection score to incorporate appropriate distribution of the data and provided the local realness to consider the features of the samples effectively.…”

Section: Anomaly Detection Methodsmentioning

confidence: 99%

“…In particular, they constructed a global disconnection score to incorporate appropriate distribution of the data and provided the local realness to consider the features of the samples effectively. Then, they determined the local outliers in smaller clusters with higher overall connectivity located in the minority class (Shi et al 2020). Chen, Wang, and Yang (2021) extended the LOF (local outlier factor) method proposed by Breunig et al (2002) and proposed a new outlier detection method, called CELOF (local outlier factor based on clustering and data extraction) (Chen, Wang, and Yang 2021).…”

Section: Anomaly Detection Methodsmentioning

confidence: 99%

See 2 more Smart Citations

A New Hybrid Under-sampling Approach to Imbalanced Classification Problems

Peng

Park

2021

Applied Artificial Intelligence

View full text Add to dashboard Cite

Among many machine learning applications, classification is one of the important tasks. Most classification algorithms have been designed under the assumption that the number of samples for each class is approximately balanced. However, if the conventional classification approaches are applied to a class imbalanced dataset, it is likely to cause misclassification and, as a result, may distort classification performance results. Thus, in this study, we consider imbalanced classification problems and adopt an efficient preprocessing technique to improve the classification performances. In particular, we focus on borderline noise and outlier samples that belong to the majority class since they may influence classification performance. For this, we propose a hybrid resampling method, called BOD-based undersampling, which is based on density-based spatial clustering of applications with noise (DBSCAN) approach as well as noise and outlier detection methods, that is, borderline noise factor (BNF) and outlierness based on neighborhood (OBN) to divide majority class samples into four distinctive categories, i.e., safe, borderline noise, rare, and outlier. Specifically, we first determine the borderline noise samples in the overlapped region using the BNF method. Secondly, we use the OBN method to detect outlier samples and apply the DBSCAN approach to cluster the samples. Based on the results obtained from the sample identification analysis, we then segregate the safe category samples which are not abnormal samples while keeping the rest of the samples as rare samples. Finally, we remove some of safe samples by using the random under-sampling (RUS) method and verify the effectiveness of the proposed algorithm through the comprehensive experimental analysis with considering several class imbalance datasets.

show abstract

Section: Anomaly Detection Methodsmentioning

confidence: 99%

Section: Anomaly Detection Methodsmentioning

confidence: 99%

Section: Anomaly Detection Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A New Hybrid Under-sampling Approach to Imbalanced Classification Problems

Peng

Park

2021

Applied Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…In general, these models are based on variables (also known as predictors) that are most likely to influence the outcome [ 26 ]. Predictive models are widely applied in various applications such as weather forecasting [ 27 , 28 , 29 ], Bayesian spam filters [ 30 , 31 , 32 , 33 ], business [ 34 , 35 , 36 , 37 ], and fraud detection [ 38 , 39 , 40 ]. Predictive models typically include a machine learning algorithm that learns certain properties from a training dataset.…”

Section: Methodsmentioning

confidence: 99%

Indirect Recognition of Predefined Human Activities

Gorjani

Proto

Vaňuš

et al. 2020

Sensors

View full text Add to dashboard Cite

The work investigates the application of artificial neural networks and logistic regression for the recognition of activities performed by room occupants. KNX (Konnex) standard-based devices were selected for smart home automation and data collection. The obtained data from these devices (Humidity, CO2, temperature) were used in combination with two wearable gadgets to classify specific activities performed by the room occupant. The obtained classifications can benefit the occupant by monitoring the wellbeing of elderly residents and providing optimal air quality and temperature by utilizing heating, ventilation, and air conditioning control. The obtained results yield accurate classification.

show abstract

“…Shi et al [14] also implement an outlier detection technique to solve class imbalance problems. Robust geodesic-based outlier detection is proposed and implemented on ten different datasets, comprising real-world and synthetic data.…”

Section: Related Workmentioning

confidence: 99%

Data Reduction for Optimizing Feature Selection in Modeling Intrusion Detection System

Iman¹,

Ahmad²

2020

IJIES

View full text Add to dashboard Cite

With the development and ease of access to internet networks, the potential for attacks and intrusions have increased. The intrusion detection system (IDS), an approach to overcome this problem, is grouped into two models: signature-based and anomaly-based. An anomaly-based IDS can be implemented by machine learning; one of the schemes in machine learning is data reduction. IDS datasets are usually obtained through a real-time process that has undefined proportional data. The purpose of data reduction is to speed up and optimize the process, improving accuracy, precision, and specifications. There are several methods to perform data reduction, one of which uses outlier detection techniques. Proper outlier detection has a positive impact on improving the classification results of machine learning. In this research, the outlier detection is done by a circle generated from the k -means clustering of all selected features. Two scenarios are designed for the evaluation: a circle generated from two points of the minimum and maximum cluster and median of all clusters. The formation of clusters conducted by k -means clustering determines the size and direction of the outlier circle so that it dynamically adjusts the distribution of data from the feature selection results. By employing the previous feature selection algorithms, the comparison is performed to evaluate the proposed method's performance. Our empirical results show that the second scenario can significantly improve the classification results in terms of accuracy, detection rate, and precision. The first and second experiments can increase the accuracy by 0.02%, and the third experiment is by 0.1%. The detection rate in the first, second, and third experiments increases by 0.01%, 0.02%, and 0.07. At the same time, precision increases by 0.04%, 0.02%, and 0.01%, correspondingly.

show abstract

Robust geodesic based outlier detection for class imbalance problem

Cited by 6 publications

References 19 publications

A New Hybrid Under-sampling Approach to Imbalanced Classification Problems

A New Hybrid Under-sampling Approach to Imbalanced Classification Problems

Indirect Recognition of Predefined Human Activities

Data Reduction for Optimizing Feature Selection in Modeling Intrusion Detection System

Contact Info

Product

Resources

About