Robust Rescaled Hinge Loss Twin Support Vector Machine for Imbalanced Noisy Classification

Wang

et al. 2022

Scientific Programming

When virtualizing large-scale images of the real world, online hashing provides an efficient scheme for fast retrieval and compact storage. It converts high-dimensional streaming data into compact binary hash codes while saving the structural characteristics between samples into the Hamming space. Existing works usually update the hashing function based on the similarity between input data, or design a codebook to assign code words for each single input sample. However, assigning code words to multiple samples while retaining the balanced similarity of the image instances is still challenging. To address this issue, we propose a novel discriminative similarity-balanced online hashing (DSBOH) framework in this work. In particular, we first obtain the Hadamard codebook that guides the generation of discriminative binary codes according to label information. Then, we maintain the correlation between the new data and the previously arrived data by the balanced similarity matrix, which is also generated by semantic information. Finally, we joined the Hadamard codebook and the balanced similarity matrix into a unified hashing function to simultaneously maintain discrimination and balanced similarity. The proposed method is optimized by an alternating optimization technique. Extensive experiments on the CIFAR-10, MNIST, and Places205 datasets demonstrate that our proposed DSBOH performs better than several state-of-the-art online hashing methods in terms of effectiveness and efficiency.

Section: Related Workmentioning

confidence: 99%

Discriminative Similarity-Balanced Online Hashing for Supervised Image Retrieval

Wang

et al. 2022

Scientific Programming

“…Twin SVM (TWSVM) has been proposed with a rescaled hinge (Rhinge) by [68] in coping with large datasets with noise and unbalanced data while maintaining the generalization ability. They tested the performance of the proposed TWSVM and Rhinge with benchmark datasets from UCI.…”

Section: Loss Function Related Workmentioning

confidence: 99%

Variants of Neural Networks: A Review

Nayef

Abdullah

Sulaiman

et al. 2022

MJCS

Machine learning (ML) techniques are part of artificial intelligence. ML involves imitating human behavior in solving different problems, such as object detection, text handwriting recognition, and image classification. Several techniques can be used in machine learning, such as Neural Networks (NN). The expansion in information technology enables researchers to collect large amounts of various data types. The challenging issue is to uncover neural network parameters suitable for object detection problems. Therefore, this paper presents a literature review of the latest proposed and developed components in neural network techniques to cope with different sizes and data types. A brief discussion is also introduced to demonstrate the different types of neural network parameters, such as activation functions, loss functions, and regularization methods. Moreover, this paper also uncovers parameter optimization methods and hyperparameters of the model, such as weight, the learning rate, and the number of iterations. From the literature, it is notable that choosing the activation function, loss function, number of neural network layers, and data size is the major factor affecting NN performance. Additionally, utilizing deep learning NN resulted in a significant improvement in model performance for a variety of issues, which became the researcher's attention.

“…The Least Square loss, i.e., L LS (u) := u 2 , is considered in [8] via a different LCQP formulation than the Hinge loss case. The Correntropy loss is defined in [9]; various compositions of this with other loss functions can be investigated, e.g., [10] for the SVM case and [11] for the imbalanced TWSVM case. One could understand the advantages of nonlinear convex loss functions for other classification methods in [12], where once again the Hinge loss is shown to be the tightest margin-based upper bound of the mis-classification loss for many classification problems.…”

Section: Background and Problem Definitionmentioning

confidence: 99%

“…The first approach to robust SVM is the use of different loss functions, with some examples given in [11], [5], [6], [7], [8], [9], [10] where the model robustness is probed by data contamination concerning the features and/or labels. The second approach relaxes the constraints of the standard optimization problem and has (2.2) rewritten as a Chance Constrained (CC) instance and/or relies on Robust optimization (RO).…”

Section: Robust Svmmentioning

confidence: 99%

Robust Classification via Support Vector Machines

Asimit¹,

Kyriakou²,

Santoni³

et al. 2021

Preprint

The loss function choice for any Support Vector Machine classifier has raised great interest in the literature due to the lack of robustness of the Hinge loss, which is the standard loss choice. In this paper, we plan to robustify the binary classifier by maintaining the overall advantages of the Hinge loss, rather than modifying this standard choice. We propose two robust classifiers under data uncertainty. The first is called Single Perturbation SVM (SP-SVM) and provides a constructive method by allowing a controlled perturbation to one feature of the data. The second method is called Extreme Empirical Loss SVM (EEL-SVM) and is based on a new empirical loss estimate, namely, the Extreme Empirical Loss (EEL), that puts more emphasis on extreme violations of the classification hyper-plane, rather than taking the usual sample average with equal importance for all hyper-plane violations. Extensive numerical investigation reveals the advantages of the two robust classifiers on simulated data and well-known real datasets.