Fully Bayesian Analysis of Relevance Vector Machine Classification With Probit Link Function for Imbalanced Data Problem

Wang, Wenyang; Sun, Dongchu; Shao, Peng; Kuang, Haibo; Sui, Cong

doi:10.1109/access.2021.3052935

Cited by 3 publications

(8 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, two more relevant methods, i.e. orthogonal least square (OLS) [11] and Bayesian sparse coding known as relevance vector machine (RVM) [12], were included in our experiments. Two sets of experiments are conducted in this section.…”

Section: Resultsmentioning

confidence: 99%

“…However, it has shown superior performance than OMP as a consequence. Relevance vector machine (RVM), as a statistical sparse coding technique, uses Bayesian model to obtain the parsimonious solutions for regression and probabilistic classification [12]. It is also called probabilistic sparse Kernel version of support vector machine (SVM) which can be used for sparse representation problems and classification.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Smooth non-negative sparse representation for face and handwritten recognition

Ghaffari

Kafaee

Abolghasemi

2021

Applied Soft Computing

View full text Add to dashboard Cite

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Smooth non-negative sparse representation for face and handwritten recognition

Ghaffari

Kafaee

Abolghasemi

2021

Applied Soft Computing

View full text Add to dashboard Cite

“… a In every cell, results are listed by AdaBoost [34], Enhanced AdaBoost [24], and Reinforced AdaBoost [24] from left to right on the top; RVM [2, 3], P‐RVM [6], and MK‐RVM [32] from left to right on the middle; Enhanced RVM (left) and Reinforced RVM (right) on the bottom. …”

Section: Numeric Studiesmentioning

confidence: 99%

“…Later, the RVM algorithm was extended to multiple-output regression and multipleoutput classification [5]. A more efficient RVM was recently proposed to employ the probit link function instead of the logistic one [6]. Relevance Vector Machine has obtained a mass of applications in recognition [7,8], detection [9,10], mechanical fault diagnosis [11,12], and electric demand forecasting [13,14].…”

Section: Introductionmentioning

confidence: 99%

Fully Bayesian analysis of the relevance vector machine classification for imbalanced data problem

Wang

Sun

et al. 2022

CAAI Trans on Intel Tech

Self Cite

View full text Add to dashboard Cite

Relevance Vector Machine (RVM) is a supervised learning algorithm extended from Support Vector Machine based on the Bayesian sparsity model. Relevance Vector Machine classification suffers from theoretical limitations and computational inefficiency mainly because there is no closed-form solution for the posterior of the weight parameters. We propose two advanced Bayesian approaches for RVM classification, namely the Enhanced RVM and the Reinforced RVM, to perfect the theoretic framework of RVM and extend the algorithm to the imbalanced data problem, which has an arresting skew in data size between classes. First, the Enhanced RVM conducts a strict Bayesian sampling process instead of the approximation method in the original one to remedy its theoretic limitations, especially the nonconvergence of the iterations. Secondly, we conjecture that the hierarchical prior makes the Reinforced RVM achieve consistent estimations of the quantities of interest compared with the non-consistent estimations of the original RVM. Consistency is necessary for RVM classification since it makes the model more stable and localises the relevant vectors more accurately in the imbalanced data problem. The two-level prior also renders the Reinforced one competitive in the imbalanced data problem by building the inner connection of parameter dimensions and alloting a more vital relevance to the small class data weight parameter. The theoretic proofs and several numeric studies demonstrate the merits of our two proposed algorithms.

show abstract

“…11 for comparisons. e first five are all the traditional ones and the sixth, PRVM [48], is designed for the imbalanced data problem, which has achieved convincing results. e last seven ones are the traditional cost-sensitive classifiers, which we have discussed in the discussion part.…”

Section: Parameters Setupmentioning

confidence: 99%

A Statistical Approach to Cost-Sensitive AdaBoost for Imbalanced Data Classification

Bei

Wang

Ren

et al. 2021

Mathematical Problems in Engineering

Self Cite

View full text Add to dashboard Cite

To address the imbalanced data problem in classification, the studies of the combination of AdaBoost, short for “Adaptive Boosting,” and cost-sensitive learning have shown convincing results in the literature. The cost-sensitive AdaBoost algorithms are practical since the “boosting” property in AdaBoost can iteratively enhance the small class of the cost-sensitive learning to solve the imbalanced data issue. However, the most available cost-sensitive AdaBoost algorithms are heuristic approaches, which are improved from the standard AdaBoost algorithm by cost-sensitively adjusting the voting weight parameters of weak classifiers or the sample updating weight parameters without strict theoretic proof. The algorithms are appended the cost-sensitive factors to focus on the high-cost and small-class samples, but they have no procedures to show the best place to add the cost factors and the cost factor value set. To complete the cost-sensitive AdaBoost algorithms’ framework, the present article has two main contributions. First, we summarize the popular cost-sensitive boosting algorithms in the literature and propose a generally comprehensive form. We name our specific one, the “AdaImC algorithm,” which is typically appliable to solve the imbalanced data classification problem with theoretic proof. Second, a statistical approach to prove the AdaImC algorithm is proposed to verify the inner relationship between the cost parameters. We show that our proposed algorithm in the machine learning field is identical to the Product of Experts (PoE) model in the statistics field. Besides, a way to determine the cost parameter value by the statistical analysis is introduced. Several numeric studies are listed finally to support our proposed algorithm.

show abstract

Fully Bayesian Analysis of Relevance Vector Machine Classification With Probit Link Function for Imbalanced Data Problem

Cited by 3 publications

References 20 publications

Smooth non-negative sparse representation for face and handwritten recognition

Smooth non-negative sparse representation for face and handwritten recognition

Fully Bayesian analysis of the relevance vector machine classification for imbalanced data problem

A Statistical Approach to Cost-Sensitive AdaBoost for Imbalanced Data Classification

Contact Info

Product

Resources

About