Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization

Shlezinger, Nir; Eldar, Yonina C.; Boyd, Stephen

doi:10.1109/access.2022.3218802

Cited by 93 publications

(20 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In model-based deep learning, DNN architectures are designed that are inspired by model-based algorithms tailored to the particular problem of interest [19]. In the context of deep receivers, the dominant model-based deep learning methodologies are deep unfolding and DNN-aided inference, which are illustrated in Fig.…”

Section: Architecturementioning

confidence: 99%

“…For each of these AI pillars, we survey candidate approaches for facilitating the operation of the deep receivers. (i) We first discuss how to design light-weight trainable architectures via model-based deep learning [19]. This methodology hinges on the principled incorporation of model-based processing, obtained from domain knowledge on optimized communication algorithms, within AI architectures.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Online Meta-Learning for Hybrid Model-Based Deep Receivers

Raviv

Park

Simeone

et al. 2023

IEEE Trans. Wireless Commun.

View full text Add to dashboard Cite

Artificial intelligence (AI) is envisioned to play a key role in future wireless technologies, with deep neural networks (DNNs) enabling digital receivers to learn to operate in challenging communication scenarios. However, wireless receiver design poses unique challenges that fundamentally differ from those encountered in traditional deep learning domains. The main challenges arise from the limited power and computational resources of wireless devices, as well as from the dynamic nature of wireless communications, which causes continual changes to the data distribution. These challenges impair conventional AI based on highly-parameterized DNNs, motivating the development of adaptive, flexible, and light-weight AI for wireless communications, which is the focus of this article. Here, we propose that AI-based design of wireless receivers requires rethinking of the three main pillars of AI: architecture, data, and training algorithms. In terms of architecture, we review how to design compact DNNs via model-based deep learning. Then, we discuss how to acquire training data for deep receivers without compromising spectral efficiency. Finally, we review efficient, reliable, and robust training algorithms via meta-learning and generalized Bayesian learning. Numerical results are presented to demonstrate the complementary effectiveness of each of the surveyed methods. We conclude by presenting opportunities for future research on the development of practical deep receivers.

show abstract

Section: Architecturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Online Meta-Learning for Hybrid Model-Based Deep Receivers

Raviv

Park

Simeone

et al. 2023

IEEE Trans. Wireless Commun.

View full text Add to dashboard Cite

show abstract

“…Meta-learning can mitigate this problem. We note that a complementary approach is to integrate data-driven and model-based approaches [110], [111], which will be briefly discussed in Section 7.…”

Section: Problem Definitionmentioning

confidence: 99%

Learning with Limited Samples -- Meta-Learning and Applications to Communication Systems

Chen¹,

Jose²,

Nikoloska³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning has achieved remarkable success in many machine learning tasks such as image classification, speech recognition, and game playing. However, these breakthroughs are often difficult to translate into real-world engineering systems because deep learning models require a massive number of training samples, which are costly to obtain in practice. To address labeled data scarcity, few-shot metalearning optimizes learning algorithms that can efficiently adapt to new tasks quickly. While meta-learning is gaining significant interest in the machine learning literature, itsThe first four authors are listed in alphabetical order. Lisha Chen is the main author of Section 2 excluding Section 2.5, as well as Sections 3, 4.6, and 7.2; Sharu Theresa Jose is the main author of Section 4; Ivana Nikoloska is the main author of Sections 2.5, 5.5 and 6.2; Sangwoo Park is the main author of Section 5 excluding Section 5.5, as well as Sections 7.1 and 7.3; Tianyi Chen is the main author of Section 3; and Osvaldo Simeone is the main author of Section 1 and Section 6.1. This monograph is based on a tutorial delivered by Tianyi Chen and Osvaldo Simeone at IEEE ICASSP 2022. Tianyi Chen and Osvaldo Simeone have supervised the writing process, and Osvaldo Simeone led the editing of the document.

show abstract

“…A second set of works focused on integrating deep learning into the basic decoder design, as a form of model-based deep learning [7]. In particular, the deep unfolding methodology was utilized in [8] to leverage data to improve BP decoding with a fixed number of iterations, and was adapted to BPbased polar decoding in [9].…”

Section: Introductionmentioning

confidence: 99%

CRC-Aided Learned Ensembles of Belief-Propagation Polar Decoders

Raviv¹,

Goldmann²,

Vayner³

et al. 2023

Preprint

View full text Add to dashboard Cite

Polar codes have promising error-correction capabilities. Yet, decoding polar codes is often challenging, particularly with large blocks, with recently proposed decoders based on list-decoding or neural-decoding. The former applies multiple decoders or the same decoder multiple times with some redundancy, while the latter family utilizes emerging deep learning schemes to learn to decode from data. In this work we introduce a novel polar decoder that combines the list-decoding with neuraldecoding, by forming an ensemble of multiple weighted beliefpropagation (WBP) decoders, each trained to decode different data. We employ the cyclic-redundancy check (CRC) code as a proxy for combining the ensemble decoders and selecting the most-likely decoded word after inference, while facilitating realtime decoding. We evaluate our scheme over a wide range of polar codes lengths, empirically showing that gains of around 0.25dB in frame-error rate could be achieved. Moreover, we provide complexity and latency analysis, showing that the number of operations required approaches that of a single BP decoder at high signal-to-noise ratios.This project has received funding from the Israeli 5G-WIN consertium. A. Goldmann, O. Vayner, T. Raviv and N. Shlezinger are with the School of ECE, Ben-Gurion University of the Negev, Beer-Sheva, Israel

show abstract

Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization

Cited by 93 publications

References 36 publications

Online Meta-Learning for Hybrid Model-Based Deep Receivers

Online Meta-Learning for Hybrid Model-Based Deep Receivers

Learning with Limited Samples -- Meta-Learning and Applications to Communication Systems

CRC-Aided Learned Ensembles of Belief-Propagation Polar Decoders

Contact Info

Product

Resources

About