DUKMSVM: A Framework of Deep Uniform Kernel Mapping Support Vector Machine for Short Text Classification

Abstract

This paper mainly deals with the problem of short text classification. There are two main contributions. Firstly, we introduce a framework of deep uniform kernel mapping support vector machine (DUKMSVM). The significant merit of this framework is that by expressing the kernel mapping function explicitly with a deep neural network, it is in essence an explicit kernel mapping instead of the traditional kernel function, and it allows better flexibility in dealing with various applications by applying different neural network structures. Secondly, to validate the effectiveness of this framework and to improve the performance of short text classification, we explicitly express the kernel mapping using bidirectional recurrent neural network (BRNN), and propose a deep bidirectional recurrent kernel mapping support vector machine (DRKMSVM) for short text classification. Experimental results on five public short text classification datasets indicate that in terms of classification accuracy, precision, recall rate and F1-score, the DRKMSVM achieves the best performance with the average values of accuracy, precision, recall rate, and F1-score of 87.23%, 86.99%, 86.13% and 86.51% respectively compared to traditional SVM, convolutional neural network (CNN), Naive Bayes (NB), and Deep Neural Mapping Support Vector Machine (DNMSVM) which applies multi-layer perceptron for kernel mapping.

Proximal Operator and Optimality Conditions for Ramp Loss SVM

“…For a given C = 0.25, we can verify that w = (0.5, 0.5) ⊤ , b = −2, u = (0, 1, 0) ⊤ , λ = (−0.25, 0, −0.25) ⊤ , ∂L r (u) = (1, 0, 1) ⊤ satisfy (12). This means (w; b; u) with λ is a KKT point of (9).…”

“…Support vector machines (SVM) were first introduced by Vapnik and Cortes [6] and have been widely applied into many fields, including text and image classification [12,25], disease detection [8,16], etc. The decision hyperplane of SVM classifier, w, x + b = 0 with w ∈ R n and b ∈ R, is trained from data set {(x i , y i ), i ∈ N m } where x i ∈ R n , y i ∈ {−1, 1} and N m := {1, 2, • • • , m} by optimizing the following problem min w∈R n ,b∈R…”

A CNN-BiGRU Text Classification Model Merging Positional Encoding and Attention Mechanism

“…where Mk is the weight matrix; Ci is the feature representation obtained through the convolution layer at time i; bk is the bias term; ui is the hidden layer representation obtained through (5). The Attention value of output data at time i is obtained through (6), and the sum of the weight of all elements is set to one through Softmax calculation; the final Attention value is obtained by (7).…”

“…When training text, mutual information method is used to check the correlation between the feature sets generated after feature selection, and the features with high correlation are combined appropriately. Reference [6] proposed an improved Support Vector Machine framework to realize short text classification. The deep neural network is used to explicitly represent the kernel mapping function instead of the traditional kernel function, which makes it more flexible in dealing with various applications.…”

Analog circuits diagnosis based on support vector machine with parameter optimization by improved NKCGWO

“…Support vector machine (SVM) is a machine leaning method based on statistic learning theory and has a good classification ability for small-sample, non-linear, high-dimension problems [20]. SVM has been widely researched and applied in many fields, such as pattern recognition [21], regression estimation [22], image recognition [23], text classification [24], fault diagnosis [25]. However, SVM classification accuracy is heavily depends on the SVM parameters, such as the penalty parameter C and the kernel parameter γ of RBF kernel function, and it is very difficult to find out the optimal SVM parameters.…”