IPMOD: An efficient outlier detection model for high-dimensional medical data streams

Yang, Yun; Fan, Chao; Chen, Liang; Xiong, Honglin

doi:10.1016/j.eswa.2021.116212

Cited by 17 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given a data set D as Table 1, each sample can be denoted by Equation (11). Each characteristic vector Fj is represented by Equation (12).…”

Section: Proposed Methodsmentioning

confidence: 99%

“…A new method called Information Entropy-Pruning Multi-dimensional Outlier Detection (IPMOD) was introduced by Yang et al [11] in which a combination of information entropy and a new index weight measurement were applied to multi-dimensional data helping to specify the effect of various attributes on data prediction. Consequently, a sliding window approach and subsequent distance measurements were used to determine if the data was outlier.…”

Section: Related Workmentioning

confidence: 99%

“…Statistical methods such as linear regression model [9] and Principle Component Analysis (PCA) [10] have traditionally been used for outlier detection. As no single outlier detection approach can be regarded as the most efficient, several other methods have been proposed in the literature [5,[11][12][13][14][15][16][17][18]. In this paper, it is attempted to detect outlier data within the test data set together with *Corresponding Author Institutional Email: hossein.nematzadeh@iau.ac.ir (H. Nematzadeh) training data set selection task.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Outlier Detection in Test Samples using Standard Deviation and Unsupervised Training Set Selection

Mohseni¹,

Nematzadeh²,

Akbarib³

et al. 2023

IJE

View full text Add to dashboard Cite

Outlier detection is a technique to identify and remove significantly different data from the more correct and consistent data in a data set. Outlier data can have negative impact on classification and clustering performance; that should be identified and removed to improve the classification efficiency. Regardless of whether a classifying technique classifies an outlier correctly, the very notion of identifying a data as outlier is of great significance. In this paper, a new approach is proposed for outlier data detection within a test data set along with unsupervised training set selection. The selected training set is used for two-step classification. After unsupervised clustering the training set, the closest cluster to a test sample is selected using the Euclidean distance measure. Then, the outlier in the test sample is identified with the concepts of standard deviation and mean value. The results showed by evaluating the distance of each sample of the test set with the new selected data set. the accuracy of the classifiers is enhanced after detection and elimination of outlier data.

show abstract

“…Given a data set D as Table 1, each sample can be denoted by Equation (11). Each characteristic vector Fj is represented by Equation (12).…”

Section: Proposed Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Outlier Detection in Test Samples using Standard Deviation and Unsupervised Training Set Selection

Mohseni¹,

Nematzadeh²,

Akbarib³

et al. 2023

IJE

View full text Add to dashboard Cite

show abstract

“…The further extension of such an approach may be also found in [ 48 ]. Apart from the above-mentioned applications, a kNN-based approach was successfully applied in [ 49 , 50 ].…”

Section: Related Workmentioning

confidence: 99%

Anomaly Detection Module for Network Traffic Monitoring in Public Institutions

Wawrowski

Bialas

Kajzer

et al. 2023

Sensors

View full text Add to dashboard Cite

It seems to be a truism to say that we should pay more and more attention to network traffic safety. Such a goal may be achieved with many different approaches. In this paper, we put our attention on the increase in network traffic safety based on the continuous monitoring of network traffic statistics and detecting possible anomalies in the network traffic description. The developed solution, called the anomaly detection module, is mostly dedicated to public institutions as the additional component of the network security services. Despite the use of well-known anomaly detection methods, the novelty of the module is based on providing an exhaustive strategy of selecting the best combination of models as well as tuning the models in a much faster offline mode. It is worth emphasizing that combined models were able to achieve 100% balanced accuracy level of specific attack detection.

show abstract

“…By detecting and adding dense neighborhoods to data objects, this algorithm progressively generates local clusters. Yang et al 18 proposed HPStream as a high-dimensional data stream clustering, which via data projection before clustering decreases the dimensionality for stream of data. The CluStream 19 framework has been proposed as an effective way to process the data stream.…”

Section: Relevant Workmentioning

confidence: 99%

RAPSAMS: Robust affinity propagation clustering on static android malware stream

Katebi

Rezakhani

Joudaki

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

This article proposed RAPSAMS, extending affinity propagation (AP) clustering to be robust in malware steaming. We use AP, which has been suggested as an approach for clustering a set of samples that by passing messages in different clusters, represents malware stream clustering. Then, by generating and adding adversarial examples, a method has been proposed to attack this clustering algorithm and try to make a robust algorithm against the proposed attack. Malware clustering has become an active research area in mobile security, in which the knowledge is being exploited in large quantities of the generated malware stream. Different procedures have been used that apply clustering to the malware's static features. This article focuses on selecting the most appropriate examples as centers for Android malware clusters and tries to add some perturbation to fool the clustering algorithm. There are two important challenges in this regard: (i) How to find the best representations for clustering.(ii) How to manage extracted patterns that include important data flow characteristics with different distributions. Malware stream clustering poses many challenges to solve this problem, including dealing with malware that is imported online, being able to process malware quickly and incrementally, dealing appropriately with time constraints, and the way that can manage important features patterns of malware stream.To examine the adaptability of the proposed methods of defense and attack, analyses were performed. The proposed approaches are tested on a couple of malware Android dataset benchmarks, namely, Contagio, Genome, and Drebin. We use permission, API, and intent features of these datasets. The obtained results from false positive rate (FPR) recognize that the amounts of the known algorithm for clustering, AP, increases to more than 50% afterward an offense occurs in a number of instances (like: Genome dataset in permission and intent features). Furthermore, by applying the label flipping attack (LFA) the accuracy measures decrease lower than 85% in all instances. Also, the proposed defense method decreased the FPR value by less than 30%, and the accuracy increased by more than 91%, in all cases. Consequently, the AP clustering will be robust against LFAs.

show abstract

IPMOD: An efficient outlier detection model for high-dimensional medical data streams

Cited by 17 publications

References 20 publications

Outlier Detection in Test Samples using Standard Deviation and Unsupervised Training Set Selection

Outlier Detection in Test Samples using Standard Deviation and Unsupervised Training Set Selection

Anomaly Detection Module for Network Traffic Monitoring in Public Institutions

RAPSAMS: Robust affinity propagation clustering on static android malware stream

Contact Info

Product

Resources

About