Compressive Sensing and Neural Networks from a Statistical Learning Perspective

DECONET: An Unfolding Network for Analysis-Based Compressed Sensing With Generalization Error Bounds

“…We also compare the performance of ADMM-DAD to that of a recent variant of ISTA-net [20]. In all experiments, ADMM-DAD outperforms the baseline and its generalization ability conforms with our theoretical results.…”

“…folding networks [20], [22], [23], [24] (see Section 2 for a brief description of a subset of them), we address the generalization analysis of a SotA ADMM-based DUN, namely ADMM-DAD [24], which deals with analysis-based CS. Towards that direction, we first exploit inherent structure of the original ADMM algorithm and impose a structural constraint on the learnable sparsifier of ADMM-DAD.…”

“…These methods incorporate a proximal mapping (3) and yield iterative schemes which, under mild assumptions, output a minimizer ẑ of (4); then the desired reconstructed signal is simply given by x = Dẑ. If D is regarded unknown and learned from a sequence of training samples, the iterations of ISTA and ADMM are interpreted as layers of neural networks; such DUNs are usually coined ISTA-nets [20], [43] and ADMM-nets [22,44], respectively. They jointly learn sparsifying transforms for the data and a decoder for CS, that is, a function reconstructing x from y.…”

“…In fact, unfolding networks based on the iterative softthresholding algorithm (ISTA [2]) and the alternating direction of multipliers method (ADMM [4]) seem to be the most popular classes of DUNs targeting the CS problem. Such networks can also learn -jointly with the decoder -a sparsifying transform for the data [20], [21], [22], [23], [24], [25], integrating that way a dictionary learning technique. Due to the advantages that the latter has offered when applied in modelbased methods [26], [27], [28], it seems intriguing to examine its effectiveness when combined with DUNs.…”

“…Nevertheless, the estimation the generalization error of DUNs is still at an early stage. Particularly, generalization error bounds are mainly provided for the class of ISTA-based unfolding networks [20], [39], [40]. To our knowledge, the generalization ability of ADMM-based DUNs is not yet explained.…”