Non-negative matrix factorization as a feature selection tool for maximum margin classifiers

Gupta, Mridula; Xiao, Jing

doi:10.1109/cvpr.2011.5995492

Cited by 33 publications

(15 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1: Generate cost function a(r) for binary-class, multi-label, multi-class tasks according to Eqs. (6), (7) and (8), respectively. 2: Calculate cost vector c i ∈ R n (1 ≤ i ≤ m) for the i-th class according to per-class FN i /FP i cost in a(r).…”

Section: Convergence Analysismentioning

confidence: 99%

Cost-Sensitive Feature Selection by Optimizing F-Measures

Luo

et al. 2018

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Feature selection is beneficial for improving the performance of general machine learning tasks by extracting an informative subset from the high-dimensional features. Conventional feature selection methods usually ignore the class imbalance problem, thus the selected features will be biased towards the majority class. Considering that F-measure is a more reasonable performance measure than accuracy for imbalanced data, this paper presents an effective feature selection algorithm that explores the class imbalance issue by optimizing F-measures. Since F-measure optimization can be decomposed into a series of cost-sensitive classification problems, we investigate the cost-sensitive feature selection by generating and assigning different costs to each class with rigorous theory guidance. After solving a series of cost-sensitive feature selection problems, features corresponding to the best F-measure will be selected. In this way, the selected features will fully represent the properties of all classes. Experimental results on popular benchmarks and challenging real-world data sets demonstrate the significance of cost-sensitive feature selection for the imbalanced data setting and validate the effectiveness of the proposed method.

show abstract

Section: Convergence Analysismentioning

confidence: 99%

Cost-Sensitive Feature Selection by Optimizing F-Measures

Luo

et al. 2018

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

show abstract

“…Since our algorithms combine feature selection, multi-kernel learning, graph regularization and NMF, we compared our algorithms with the following relevant methods: the original NMF (Lee & Seung, 2000), the graph-regularized NMF (GNMF) (Cai et al, 2011), the kernel NMF (NMF K ) (Lee, Cichocki, & Choi, 2009), the NMF with multi-kernels (NMF MK ) and the NMF with feature selection (NMF FS ) (Das Gupta & Xiao, 2011). In total seven different methods were compared on this data set.…”

Section: Resultsmentioning

confidence: 99%

Feature selection and multi-kernel learning for adaptive graph regularized nonnegative matrix factorization

Wang

Huang

Sun

et al. 2015

Expert Systems with Applications

View full text Add to dashboard Cite

a b s t r a c tNonnegative matrix factorization (NMF), a popular part-based representation technique, does not capture the intrinsic local geometric structure of the data space. Graph regularized NMF (GNMF) was recently proposed to avoid this limitation by regularizing NMF with a nearest neighbor graph constructed from the input data set. However, GNMF has two main bottlenecks. First, using the original feature space directly to construct the graph is not necessarily optimal because of the noisy and irrelevant features and nonlinear distributions of data samples. Second, one possible way to handle the nonlinear distribution of data samples is by kernel embedding. However, it is often difficult to choose the most suitable kernel. To solve these bottlenecks, we propose two novel graph-regularized NMF methods, AGNMF FS and AGNMF MK , by introducing feature selection and multiple-kernel learning to the graph regularized NMF, respectively. Instead of using a fixed graph as in GNMF, the two proposed methods learn the nearest neighbor graph that is adaptive to the selected features and learned multiple kernels, respectively. For each method, we propose a unified objective function to conduct feature selection/multi-kernel learning, NMF and adaptive graph regularization simultaneously. We further develop two iterative algorithms to solve the two optimization problems. Experimental results on two challenging pattern classification tasks demonstrate that the proposed methods significantly outperform state-of-the-art data representation methods.

show abstract

“…However, simultaneously achieving feature learning and learning to rank has not been reported in the literature. The joint optimization principle is introduced in [9], [36]. Gupta et al [9] pursue a discriminative decomposition by coupling NMF objective with a maximum margin classifier.…”

Section: Related Workmentioning

confidence: 99%

“…The joint optimization principle is introduced in [9], [36]. Gupta et al [9] pursue a discriminative decomposition by coupling NMF objective with a maximum margin classifier. Fang et al [36] present a new classification framework using the multi-label correlation information to address the problem of simultaneously combining multiple feature views and maximum margin classification.…”

Section: Related Workmentioning

confidence: 99%

“…Specifically, as shown in Fig. 1, the high-order interaction information on data samples is captured by learning a structural SVM ranking model, while the linear feature learning task is associated with a simple and effective matrix factorization discovering a low-rank structural representation [9]. In theory, these two parts are mutually reinforced by iteratively solving an alternating optimization problem with respect to feature learning and ranking model construction.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Joint Structural Learning to Rank with Deep Linear Feature Learning

Zhao

Zhang

2015

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Multimedia information retrieval usually involves two key modules including effective feature representation and ranking model construction. Most existing approaches are incapable of well modeling the inherent correlations and interactions between them, resulting in the loss of the latent consensus structure information. To alleviate this problem, we propose a learning to rank approach that simultaneously obtains a set of deep linear features and constructs structure-aware ranking models in a joint learning framework. Specifically, the deep linear feature learning corresponds to a series of matrix factorization tasks in a hierarchical manner, while the learning-to-rank part concentrates on building a ranking model that effectively encodes the intrinsic ranking information by structural SVM learning. Through a joint learning mechanism, the two parts are mutually reinforced in our approach, and meanwhile their underlying interaction relationships are implicitly reflected by solving an alternating optimization problem. Due to the intrinsic correlations among different queries (i.e., similar queries for similar ranking lists), we further formulate the learning-to-rank problem as a multi-task problem, which is associated with a set of mutually related queryspecific learning-to-rank subproblems. For computational efficiency and scalability, we design a MapReduce-based parallelization approach to speed up the learning processes. Experimental results demonstrate the efficiency, effectiveness, and scalability of the proposed approach in multimedia information retrieval.

show abstract

Non-negative matrix factorization as a feature selection tool for maximum margin classifiers

Cited by 33 publications

References 20 publications

Cost-Sensitive Feature Selection by Optimizing F-Measures

Cost-Sensitive Feature Selection by Optimizing F-Measures

Feature selection and multi-kernel learning for adaptive graph regularized nonnegative matrix factorization

Joint Structural Learning to Rank with Deep Linear Feature Learning

Contact Info

Product

Resources

About