Majorization Minimization Technique for Optimally Solving Deep Dictionary Learning

Abstract

The total number of words of the manuscript, including entire text from title page to figure legends: 5050The number of words of the abstract: 199The number of figures: 7The number of tables: 2 Deep Dictionary Learning Page nr. 2 of 20 2The concept of deep dictionary learning has been recently proposed. Unlike shallow dictionary learning which learns single level of dictionary to represent the data, it uses multiple layers of dictionaries. So far, the problem could only be solved in a greedy fashion; this was achieved by learning a single layer of dictionary in each stage where the coefficients from the previous layer acted as inputs to the subsequent layer (only the first layer used the training samples as inputs). This was not optimal; there was feedback from shallower to deeper layers but not the other way.This work proposes an optimal solution to deep dictionary learning whereby all the layers of dictionaries are solved simultaneously. We employ the Majorization Minimization approach.Experiments have been carried out on benchmark datasets; it shows that optimal learning indeed improves over greedy piecemeal learning. Comparison with other unsupervised deep learning tools (stacked denoising autoencoder, deep belief network, contractive autoencoder and K-sparse autoencoder) show that our method supersedes their performance both in accuracy and speed.

Convolutional Dictionary Pair Learning Network for Image Representation Learning.pdf

“…MNIST Fashion-MNIST SRC [5] 84.61% 79.86% DKSVD [7] 84.69% 78.38% LC-KSVD1 [1] 84.72% 78.50% LC-KSVD2 [1] 85.88% 79.27% DLSI [9] 88.39% 79.48% FDDL [8] 87.93% 80.67% DPL [10] 90.08% 83.50% LRSDL [19] 87.80% 81.99% ADDL [12] 88.90% 82.10% DDL [22] 98.33% -SCN-4 [6] 97.98% 88.73% DDLCN (100-100) [21] 98.55% -CDPL-Net (no DPL layers) 98.64%…”

Convolutional Dictionary Pair Learning Network for Image Representation Learning.pdf

“…We mainly evaluate our CDPL-Net for image representation and classification. The performance of CDPL-Net is mainly compared with several traditional DL methods including the sparse representation based classification (SRC) [5], DLSI [9], D-KSVD [7], LC-KSVD [1], FDDL [8], DPL [10], LRSDL [19] and ADDL [12], and four related deep learning models, including deep sparse coding network (SCN) [6], DDL [22], and DDLCN [21]. For image representation and classification on each database, we split it into a training set and a test set.…”

“…MNIST Fashion-MNIST SRC [5] 84.61% 79.86% DKSVD [7] 84.69% 78.38% LC-KSVD1 [1] 84.72% 78.50% LC-KSVD2 [1] 85.88% 79.27% DLSI [9] 88.39% 79.48% FDDL [8] 87.93% 80.67% DPL [10] 90.08% 83.50% LRSDL [19] 87.80% 81.99% ADDL [12] 88.90% 82.10% DDL [22] 98.33% -SCN-4 [6] 97.98% 88.73% DDLCN (100-100) [21] 98.55% -CDPL-Net (no DPL layers) 98.64% 87.46% Our CDPL-Net 98.98% 90.69% Figure 7. Accuracy on Fashion-MNIST with different batch sizes.…”

“…Tang et al proposed a deep Micro-dictionary learning and coding network (DDLCN) for image representation and classification [21], which simply includes the feature extraction layer and multiple DL layers together. Singhal et al have also proposed a deep dictionary leaning network DDL for image classification [22], which performs multiple DL for feature coding, followed by reconstruction error minimization. Wang et al have proposed an object detection system by unifying the dictionary pair learning with convolutional feature learning [3], where the learned dictionary is used as a classifier yet only one layer.…”

AEAO: Auto Encoder with Adam Optimizer Method for Efficient Document Indexing of Big Data

“…20-News group data sets [53][54][55][56][57][58] (http://qwone.com/~jason/20Newsgroups/) is used in this research for evaluation of the proposed method. This data set consists of approximately 20,000 documents and it is one of the common data set used for the document indexing technique.…”