Deep Micro-Dictionary Learning and Coding Network

Abstract

In this paper, we propose a novel Deep Micro-Dictionary Learning and Coding Network (DDLCN). DDLCN has most of the standard deep learning layers (pooling, fully, connected, input/output, etc.) but the main difference is that the fundamental convolutional layers are replaced by novel compound dictionary learning and coding layers. The dictionary learning layer learns an over-complete dictionary for the input training data. At the deep coding layer, a locality constraint is added to guarantee that the activated dictionary bases are close to each other. Next, the activated dictionary atoms are assembled together and passed to the next compound dictionary learning and coding layers. In this way, the activated atoms in the first layer can be represented by the deeper atoms in the second dictionary. Intuitively, the second dictionary is designed to learn the finegrained components which are shared among the input dictionary atoms. In this way, a more informative and discriminative low-level representation of the dictionary atoms can be obtained. We empirically compare the proposed DDLCN with several dictionary learning methods and deep learning architectures. The experimental results on four popular benchmark datasets demonstrate that the proposed DDLCN achieves competitive results compared with state-of-the-art approaches.ing methods focus on extracting semantic features via a deep network. So far most studies in dictionary learning employ a shallow (single layer) architecture, e.g., currently popular dictionary learning techniques are K-SVD [1], Discriminative K-SVD (D-KSVD) [57] and Label Consistent K-SVD (LC-KSVD) [20] which decompose the training data into a dense basis and sparse coefficients. In addition, both Local Coordinate Coding (LCC) [55, 54] and arXiv:1809.04185v2 [cs.CV]

Convolutional Dictionary Pair Learning Network for Image Representation Learning.pdf

“…MNIST Fashion-MNIST SRC [5] 84.61% 79.86% DKSVD [7] 84.69% 78.38% LC-KSVD1 [1] 84.72% 78.50% LC-KSVD2 [1] 85.88% 79.27% DLSI [9] 88.39% 79.48% FDDL [8] 87.93% 80.67% DPL [10] 90.08% 83.50% LRSDL [19] 87.80% 81.99% ADDL [12] 88.90% 82.10% DDL [22] 98.33% -SCN-4 [6] 97.98% 88.73% DDLCN (100-100) [21] 98.55% -CDPL-Net (no DPL layers) 98.64%…”

Convolutional Dictionary Pair Learning Network for Image Representation Learning.pdf

“…We mainly evaluate our CDPL-Net for image representation and classification. The performance of CDPL-Net is mainly compared with several traditional DL methods including the sparse representation based classification (SRC) [5], DLSI [9], D-KSVD [7], LC-KSVD [1], FDDL [8], DPL [10], LRSDL [19] and ADDL [12], and four related deep learning models, including deep sparse coding network (SCN) [6], DDL [22], and DDLCN [21]. For image representation and classification on each database, we split it into a training set and a test set.…”

“…MNIST Fashion-MNIST SRC [5] 84.61% 79.86% DKSVD [7] 84.69% 78.38% LC-KSVD1 [1] 84.72% 78.50% LC-KSVD2 [1] 85.88% 79.27% DLSI [9] 88.39% 79.48% FDDL [8] 87.93% 80.67% DPL [10] 90.08% 83.50% LRSDL [19] 87.80% 81.99% ADDL [12] 88.90% 82.10% DDL [22] 98.33% -SCN-4 [6] 97.98% 88.73% DDLCN (100-100) [21] 98.55% -CDPL-Net (no DPL layers) 98.64% 87.46% Our CDPL-Net 98.98% 90.69% Figure 7. Accuracy on Fashion-MNIST with different batch sizes.…”

“…Hu et al developed a nonlinear DL method for image classification [14]. Tang et al proposed a deep Micro-dictionary learning and coding network (DDLCN) for image representation and classification [21], which simply includes the feature extraction layer and multiple DL layers together. Singhal et al have also proposed a deep dictionary leaning network DDL for image classification [22], which performs multiple DL for feature coding, followed by reconstruction error minimization.…”

When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition with Limited Data

“…Part of this work has been published in [25]. The additional contributions are: 1) We present a more detailed analysis by including recently published works about deep dictionary learning.…”