Ramp Loss based robust one-class SVM

Xiao, Yingchao; Wang, Huangang; Wang, Xu

doi:10.1016/j.patrec.2016.11.016

Cited by 37 publications

(23 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhu et al [31] and Cha et al [32] employed nearest neighbor information to compute the relevant weights. Furthermore, Xiao et al [34] recently introduced the ramp loss function into model training to effectively limit the negative influence of anomalies.…”

Section: Robustnessmentioning

confidence: 99%

See 1 more Smart Citation

Diversity Aware-Based Sequential Ensemble Learning for Robust Anomaly Detection

Chen

2020

IEEE Access

View full text Add to dashboard Cite

Anomaly detection has become a popular topic in many domains because anomalies can provide valuable information. Recently, ensemble learning has been applied to improve the generalization ability of existing anomaly detection methods. In an anomaly ensemble framework, diversity is essential for building a powerful ensemble method to obtain impressive performance. However, most studies use heuristic techniques to improve the diversity of ensembles, and it generally leads to limited diversity. To obtain improved diversity, we propose a diversity aware-based sequential ensemble method (called D-SEM) for anomaly detection. Specifically, our proposed method divides the ensemble diversity into two parts: sample diversity and model diversity. For sample diversity, we introduce the subsampling technique to implement preliminary generation of diverse datasets for training. For model diversity, we design an ensemble-based optimization model to learn base classifiers with improved diversity. Furthermore, an unsupervised diversity measure is proposed to quantitatively assess diversity and an anomaly pruning strategy is utilized to successively eliminate pseudo-anomalies. Based on the inclusion of sample diversity and model diversity, the proposed D-SEM method obtains better generalization ability for anomaly detection. The experimental results based on real-world datasets suggest that the proposed method has superior performance compared with various state-of-the-art methods.

show abstract

Section: Robustnessmentioning

confidence: 99%

“…For the generation of model diversity, we improve the OCSVM as a base classifier. The general equation of the OCSVM [34] is as follows:…”

Section: ) Problem Analysismentioning

confidence: 99%

Diversity Aware-Based Sequential Ensemble Learning for Robust Anomaly Detection

Chen

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Experimental evaluations on several data sets demonstrate that the our proposed ensemble loss function significantly improves the performance of a simple regressor in comparison with state-of-the-art methods.Loss functions are fundamental components of machine learning systems and are used to train the parameters of the learner model. Since standard training methods aim to determine the parameters that minimize the average value of the loss given an annotated training set, loss functions are crucial for successful trainings [49,55]. Bayesian estimators are obtained by minimizing the expected loss function.…”

mentioning

confidence: 99%

“…The 0-1 loss function is known as a robust loss because it assigns value 1 to all misclassified samples -including outliers -and thus an outlier does not influence the decision function, leading to a robust learner. On the other hand, the 0-1 loss penalizes all misclassified samples equally with value 1, and since it does not enhance the margin, it cannot be an appropriate choice for applications with margin importance [49].The Ramp loss function, as another type of loss functions, is defined similarly to the 0-1 loss function with the only difference that ramp loss functions also penalize some correct samples, those with small margins. This minor difference makes the Ramp loss function appropriate for applications with margin importance [43,49].…”

mentioning

confidence: 99%

“…On the other hand, the 0-1 loss penalizes all misclassified samples equally with value 1, and since it does not enhance the margin, it cannot be an appropriate choice for applications with margin importance [49].The Ramp loss function, as another type of loss functions, is defined similarly to the 0-1 loss function with the only difference that ramp loss functions also penalize some correct samples, those with small margins. This minor difference makes the Ramp loss function appropriate for applications with margin importance [43,49]. On the downside, the Ramp loss function is not differentiable, and hence not suitable for optimization purposes.The Sigmoid loss function is almost the same as 0-1 loss functions, except that it is differentiable, which in turn makes optimization significantly easier.…”

mentioning

confidence: 99%

See 1 more Smart Citation

relf: robust regression extended with ensemble loss function

2018

View full text Add to dashboard Cite

Ensemble techniques are powerful approaches that combine several weak learners to build a stronger one. As a meta-learning framework, ensemble techniques can easily be applied to many machine learning methods. Inspired by ensemble techniques, in this paper we propose an ensemble loss functions applied to a simple regressor. We then propose a half-quadratic learning algorithm in order to find the parameter of the regressor and the optimal weights associated with each loss function. Moreover, we show that our proposed loss function is robust in noisy environments. For a particular class of loss functions, we show that our proposed ensemble loss function is Bayes consistent and robust. Experimental evaluations on several data sets demonstrate that the our proposed ensemble loss function significantly improves the performance of a simple regressor in comparison with state-of-the-art methods.Loss functions are fundamental components of machine learning systems and are used to train the parameters of the learner model. Since standard training methods aim to determine the parameters that minimize the average value of the loss given an annotated training set, loss functions are crucial for successful trainings [49,55]. Bayesian estimators are obtained by minimizing the expected loss function. Different loss functions lead to different Optimum Bayes with possibly different characteristics. Thus, in each environment the choice of the underlying loss function is important, as it will impact the performance [44,48].Lettingθ denote the estimated parameter of a correct parameter θ, the loss function L(θ, θ) is a positive function which assigns a loss value to each estimation, indicating how inaccurate the estimation is [42]. Loss functions assign a value to each sample, indicating how much that sample contributes to solving the optimization problem. Each loss function comes with its own advantages and disadvantages. In order to put our results in context, we start by reviewing three popular loss functions (0-1, Ramp and Sigmoid) and we will give an overview of their advantages and disadvantages.Loss functions assign a value to each sample representing how much that sample contributes to solving the optimization problem. If an outlier is given a very large value by the loss function, it might dramatically affect the decision function [18]. The 0-1 loss function is known as a robust loss because it assigns value 1 to all misclassified samples -including outliers -and thus an outlier does not influence the decision function, leading to a robust learner. On the other hand, the 0-1 loss penalizes all misclassified samples equally with value 1, and since it does not enhance the margin, it cannot be an appropriate choice for applications with margin importance [49].The Ramp loss function, as another type of loss functions, is defined similarly to the 0-1 loss function with the only difference that ramp loss functions also penalize some correct samples, those with small margins. This minor difference makes the Ramp loss function appro...

show abstract

OAAE: Optimized adaptive anomaly detection ensemble—Base model boosting by parameter optimization

Bii

2021

Engineering Reports

View full text Add to dashboard Cite

Irregular data or anomalies may occur due to human error, miscalculation, or malicious system behavior. The detection of anomalies is a difficult task that requires the use of multiple strategic methods and models. The ideal detection model should assess the strengths and optimize the results of its base models before making final decisions. This task of optimizing the results of the base models contributes to the generation of more accurate results overall. This work presents an optimized adaptive anomaly detection ensemble using heterogeneous algorithms. In the first stage, it adaptively boosts the outcomes of preceding models by weighting their decisions and finding high‐performance ones, and in the second stage, it optimizes the base models by score margin maximization, which enlarges the contrast between the scores of the anomalies and other data prior to fusion to improve detection accuracy. To validate the model, baselines and test results from 10 benchmark datasets are compared. To assess its effectiveness in terms of distinguishing anomalies, the proposed model is tested and evaluated. The analyzed data are presented as cross‐tabulations, with detailed explanations and interpretations. The experiments show an improvement in results even when the least of anomalies (single cases up to 10%) are used.

show abstract

Ramp Loss based robust one-class SVM

Cited by 37 publications

References 19 publications

Diversity Aware-Based Sequential Ensemble Learning for Robust Anomaly Detection

Diversity Aware-Based Sequential Ensemble Learning for Robust Anomaly Detection

relf: robust regression extended with ensemble loss function

OAAE: Optimized adaptive anomaly detection ensemble—Base model boosting by parameter optimization

Contact Info

Product

Resources

About