Learning to decode linear codes using deep learning

Nachmani, Eliya; Be'ery, Y.; Burshtein, David

doi:10.1109/allerton.2016.7852251

Cited by 450 publications

(484 citation statements)

References 19 publications

Supporting

Mentioning

473

Contrasting

Unclassified

Order By: Relevance

“…Among future directions, it is worth considering to combine other neural decodes with MIND, such as neural LDPC [29] [30] and Polar [32] decoders. Beyond neural decoder design, MAML can also be applied to Channel Autoencoder [27] design, which deals with designing adaptive encoder and decoder.…”

Section: Discussionmentioning

confidence: 99%

“…Designing neural decoder for several classes of codes such as LDPC codes, Polar codes and Turbo codes with versatile deep neural networks has seen a growing interest within the channel coding community. Imitating Belief Propagation (BP) algorithm via learnable neural networks shows promising performance for High-Density Parity-Check (HDPC) codes and LDPC codes [29] [30] and Polar codes [31] [32]. Near optimal performance of Convolutional Code and Turbo Code under AWGN channel is achieved via Recurrent Neural Networks (RNN) for arbitrary block lengths [33], which also shows robust and adaptive performance under non-AWGN setups.…”

Section: B Prior Art : Neural Decodingmentioning

confidence: 99%

See 1 more Smart Citation

MIND: Model Independent Neural Decoder

Jiang

Kim

Asnani

et al. 2019

2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC)

View full text Add to dashboard Cite

Standard decoding approaches rely on model-based channel estimation methods to compensate for varying channel effects, which degrade in performance whenever there is a model mismatch. Recently proposed Deep learning based neural decoders address this problem by leveraging a model-free approach via gradient-based training. However, they require large amounts of data to retrain to achieve the desired adaptivity, which becomes intractable in practical systems.In this paper, we propose a new decoder: Model Independent Neural Decoder (MIND), which builds on the top of neural decoders and equips them with a fast adaptation capability to varying channels. This feature is achieved via the methodology of Model-Agnostic Meta-Learning (MAML). Here the decoder: (a) learns a "good" parameter initialization in the meta-training stage where the model is exposed to a set of archetypal channels and (b) updates the parameter with respect to the observed channel in the meta-testing phase using minimal adaptation data and pilot bits. Building on top of existing state-ofthe-art neural Convolutional and Turbo decoders, MIND outperforms the static benchmarks by a large margin and shows minimal performance gap when compared to the neural (Convolutional or Turbo) decoders designed for that particular channel. In addition, MIND also shows strong learning capability for channels not exposed during the meta training phase.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: B Prior Art : Neural Decodingmentioning

confidence: 99%

MIND: Model Independent Neural Decoder

Jiang

Kim

Asnani

et al. 2019

2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC)

View full text Add to dashboard Cite

show abstract

“…The image or the attribute intermediate hash codes generated by ADCMH can be considered to be corrupted codewords within a certain distance d of a correct codeword of an ECC. Recently, some excellent neural network based ECC decoders have been proposed [26], [27], [28], which have achieved close to the maximum likelihood (ML) performance for an ECC. These methods can be leveraged to generate high-quality and efficient hash codes.…”

Section: Stage 1(b): Training Neural Error Correcting Decodermentioning

confidence: 99%

“…In this process, the attribute hash code and image hash code of the same subject are forced to map to the same codeword, thereby reducing the distance of the corresponding hash codes. This brings more relevant facial images from the gallery closer to the attribute query, which leads to an improved retrieval performance.Recent work has shown that the same kinds of neural network architectures used for classification can also be used to decode ECC codes [26], [27], [28]. Motivated by this, we have used a neural error-correction decoder (NECD) [26] as an ECC decoder to improve the cross-modal retrieval efficiency.…”

mentioning

confidence: 99%

“…This brings more relevant facial images from the gallery closer to the attribute query, which leads to an improved retrieval performance.Recent work has shown that the same kinds of neural network architectures used for classification can also be used to decode ECC codes [26], [27], [28]. Motivated by this, we have used a neural error-correction decoder (NECD) [26] as an ECC decoder to improve the cross-modal retrieval efficiency. The NECD is a non-fully connected neural network architecture based on the belief propagation (BP) algorithm, which is a notable decoding technique applied for error-correcting codes.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Error-Corrected Margin-Based Deep Cross-Modal Hashing for Facial Image Retrieval

Taherkhani

Talreja

Valenti

et al. 2020

IEEE Trans. Biom. Behav. Identity Sci.

View full text Add to dashboard Cite

Cross-modal hashing facilitates mapping of heterogeneous multimedia data into a common Hamming space, which can be utilized for fast and flexible retrieval across different modalities. In this paper, we propose a novel cross-modal hashing architecture-deep neural decoder cross-modal hashing (DNDCMH), which uses a binary vector specifying the presence of certain facial attributes as an input query to retrieve relevant face images from a database. The DNDCMH network consists of two separate components: an attribute-based deep cross-modal hashing (ADCMH) module, which uses a margin (m)-based loss function to efficiently learn compact binary codes to preserve similarity between modalities in the Hamming space, and a neural error correcting decoder (NECD), which is an error correcting decoder implemented with a neural network. The goal of NECD network in DNDCMH is to error correct the hash codes generated by ADCMH to improve the retrieval efficiency. The NECD network is trained such that it has an error correcting capability greater than or equal to the margin (m) of the margin-based loss function. This results in NECD can correct the corrupted hash codes generated by ADCMH up to the Hamming distance of m. We have evaluated and compared DNDCMH with state-of-the-art cross-modal hashing methods on standard datasets to demonstrate the superiority of our method.! DCMH [22], the inter-modal triplet embedding loss encourages the heterogeneous correlation across different modalities, and the intra-modal triplet loss encodes the discriminative power of the hash codes. Moreover, a regularization loss is used to apply adjacency consistency to ensure that the hash codes can keep the original similarities in Hamming space. However, in margin-based loss functions, some of the instances of different modalities of the same subject may not be close enough in Hamming space to guarantee all the correct retrievals. Therefore, it is important to bring the different modalities of the same subject closer to each other in Hamming space to improve the retrieval efficiency.In this work, we observe that in addition to the regular DCMH techniques [13], [24], [25], which exploit entropy maximization and quantization losses in the objective function of the DCMH, an error-correcting code (ECC) decoder can be used as an additional component to compensate for the heterogeneity gap and reduce the Hamming distance between the different modalities of the same subject in order to improve the cross-modal retrieval efficiency. We presume that the hash code generated by DCMH is a binary vector that is within a certain distance from a codeword of an ECC. When the hash code generated by DCMH is passed through an ECC decoder, the closest codeword to this hash code is found, which can be used as a final hash code for the retrieval process. In this process, the attribute hash code and image hash code of the same subject are forced to map to the same codeword, thereby reducing the distance of the corresponding hash codes. This brings more relevant facial images ...

show abstract