Hierarchical feature extraction by multi-layer non-negative matrix factorization network for classification task

Song, Hyun Ah; Kim, Bo-Kyeong; Xuan, Thanh Luong; Lee, Soo Young

doi:10.1016/j.neucom.2014.08.095

Cited by 42 publications

(19 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experiments for the face recognition rate with different values of r based on threshold SNMF [24], CNMF [21], multi-layer NMF [22], of multiplicative iterative rules, PCA [12] and the SWNMF based on the new additive iteration rules proposed in this paper are executed on the JAFEE database and ORL database, respectively. The results of comparison experiments are shown in Figures 3-6.…”

Section: Comparision Of Swnmf With Multiple Iteration Nmf Methods Andmentioning

confidence: 99%

“…Therefore, it has been widely used in image, voice, video and other non-negative datasets to extract the features [18,19]. In recent years, many improved NMF algorithms have been proposed, such as SNMF (sparse nonnegative matrix factorization) method [20], the CNMF (convex nonnegative matrix factorization) method [21], and the multi-layer NMF method [22]. The SNMF method considers the redundant information hidden in the complex data by adding the sparse constraints in the iteration rule.…”

Section: Introductionmentioning

confidence: 99%

“…Meanwhile, the respective weights of important information and less important information of the face data are not considered, which will result in the hierarchy of the extracted feature not being clear. Thus, by adding a weight matrix in the iteration rules to improve the hierarchical expression of local features relative to the overall features and improving the iteration step size, a new sparse weighted non-negative matrix factorization (SWNMF) method based on the new iterative step size is proposed in this paper, whose recognition rate is higher than the NMF method, the SNMF [24] method, CNMF method [21], and multi-layer NMF method [22]. Meanwhile, the deep learning methods are also widely used in face feature extraction.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Research of SWNMF with New Iteration Rules for Facial Feature Extraction and Recognition

Zhou

2019

Symmetry

View full text Add to dashboard Cite

Weighted nonnegative matrix factorization (WNMF) is a technology for feature extraction, which can extract the feature of face dataset, and then the feature can be recognized by the classifier. To improve the performance of WNMF for feature extraction, a new iteration rule is proposed in this paper. Meanwhile, the base matrix U is sparse based on the threshold, and the new method is named sparse weighted nonnegative matrix factorization (SWNMF). The new iteration rules are based on the smaller iteration steps, thus, the search is more precise, therefore, the recognition rate can be improved. In addition, the sparse method based on the threshold is adopted to update the base matrix U, which can make the extracted feature more sparse and concentrate, and then easier to recognize. The SWNMF method is applied on the ORL and JAFEE datasets, and from the experiment results we can find that the recognition rates are improved extensively based on the new iteration rules proposed in this paper. The recognition rate of new SWNMF method reached 98% for ORL face database and 100% for JAFEE face database, respectively, which are higher than the PCA method, the sparse nonnegative matrix factorization (SNMF) method, the convex non-negative matrix factorization (CNMF) method and multi-layer NMF method.

show abstract

Section: Comparision Of Swnmf With Multiple Iteration Nmf Methods Andmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Research of SWNMF with New Iteration Rules for Facial Feature Extraction and Recognition

Zhou

2019

Symmetry

View full text Add to dashboard Cite

show abstract

“…where λ l e T l e l H and λ l · e T l e l 2 in (24) are removed if l = L, meaning that we just controled the sparsity of H L to obtain a high-level final presentation. After pre-training each layer separately, we fine-tuned the weights of all layers as well as the final representation with initial approximation points of W l , H L to reduce the total reconstruction error of (21). In particular, for a specific l and a factor matrix W l , let's consider the model in (15) where, W :…”

Section: Convergence Analysismentioning

confidence: 99%

Sparse deep nonnegative matrix factorization

Guo

Zhang

2020

Big Data Min. Anal.

View full text Add to dashboard Cite

Nonnegative matrix factorization is a powerful technique to realize dimension reduction and pattern recognition through single-layer data representation learning. Deep learning, however, with its carefully designed hierarchical structure, is able to combine hidden features to form more representative features for pattern recognition. In this paper, we proposed sparse deep nonnegative matrix factorization models to analyze complex data for more accurate classification and better feature interpretation. Such models are designed to learn localized features or generate more discriminative representations for samples in distinct classes by imposing L1-norm penalty on the columns of certain factors. By extending one-layer model into multi-layer one with sparsity, we provided a hierarchical way to analyze big data and extract hidden features intuitively due to nonnegativity. We adopted the Nesterov's accelerated gradient algorithm to accelerate the computing process with the convergence rate of O(1/k 2 ) after k steps iteration. We also analyzed the computing complexity of our framework to demonstrate their efficiency. To improve the performance of dealing with linearly inseparable data, we also considered to incorporate popular nonlinear functions into this framework and explored their performance. We applied our models onto two benchmarking image datasets, demonstrating our models can achieve competitive or better classification performance and produce intuitive interpretations compared with the typical NMF and competing multi-layer models.

show abstract

“…Cichocki et al propose a multilayer NMF algorithm with multi-start initializations, which is also a deep factorization of the coefficient matrix [30], [31]. Song et al extended the nsNMF algorithm that can learn sparse features to an algorithm with a multilayered structure [32]. In 2017, Trigeorgis et al extended the SN-MF algorithm to a multilayer structure and proposed the deep semi NMF (DSNMF) algorithm [33].…”

Section: Introductionmentioning

confidence: 99%

Deep Non-Negative Matrix Factorization Architecture Based on Underlying Basis Images Learning

Zhao

Wang

Pei

2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

The non-negative matrix factorization (NMF) algorithm represents the original image as a linear combination of a set of basis images. This image representation method is in line with the idea of "parts constitute a whole" in human thinking. The existing deep NMF performs deep factorization on the coefficient matrix. In these methods, the basis images used to represent the original image is essentially obtained by factorizing the original images once. To extract features reflecting the deep localization characteristics of images, a novel deep NMF architecture based on underlying basis images learning is proposed for the first time. The architecture learns the underlying basis images by deep factorization on the basis images matrix. The deep factorization architecture proposed in this paper has strong interpretability. To implement this architecture, this paper proposes a deep non-negative basis matrix factorization algorithm to obtain the underlying basis images. Then, the objective function is established with an added regularization term, which directly constrains the basis images matrix to obtain the basis images with good local characteristics, and a regularized deep non-negative basis matrix factorization algorithm is proposed. The regularized deep nonlinear non-negative basis matrix factorization algorithm is also proposed to handle pattern recognition tasks with complex data. This paper also theoretically proves the convergence of the algorithm. Finally, the experimental results show that the deep NMF architecture based on the underlying basis images learning proposed in this paper can obtain better recognition performance than the other state-of-the-art methods.

show abstract

Hierarchical feature extraction by multi-layer non-negative matrix factorization network for classification task

Cited by 42 publications

References 13 publications

Research of SWNMF with New Iteration Rules for Facial Feature Extraction and Recognition

Research of SWNMF with New Iteration Rules for Facial Feature Extraction and Recognition

Sparse deep nonnegative matrix factorization

Deep Non-Negative Matrix Factorization Architecture Based on Underlying Basis Images Learning

Contact Info

Product

Resources

About