Nested compression of convolutional neural networks with Tucker-2 decomposition

Zdunek, Rafał; Gabor, Mateusz

doi:10.1109/ijcnn55064.2022.9892959

Cited by 2 publications

(2 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is also a possibility to use the weights of the pre-trained network and factorize it into the values of smaller tensors using, for example, higher-order singular-value decomposition (HOSVD) [ 83 ]. In [ 84 ], the above-mentioned techniques were applied to well-known architectures (ResNet [ 85 ], VGG [ 86 ]) for image classification. These architectures contain both convolutional and fully connected layers.…”

Section: Techniques For the Reduction In Computational And Memory Req...mentioning

confidence: 99%

“…These architectures contain both convolutional and fully connected layers. In the case of fully connected layers, truncated singular-value decomposition was used to factorize their weight matrices while tensors representing convolutional layers were factorized using Tucker-2 [ 87 ] and nested Tucker-2 decompositions [ 84 ]. The results obtained for the image classification task for MNIST [ 88 ] and CIFAR-10 [ 89 ] showed a small decrease in accuracy while reducing both the memory requirement and computational complexity.…”

Section: Techniques For the Reduction In Computational And Memory Req...mentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Low-Latency DNN-Based Speech Enhancement

Drgas

2023

Sensors

View full text Add to dashboard Cite

This paper presents recent advances in low-latency, single-channel, deep neural network-based speech enhancement systems. The sources of latency and their acceptable values in different applications are described. This is followed by an analysis of the constraints imposed on neural network architectures. Specifically, the causal units used in deep neural networks are presented and discussed in the context of their properties, such as the number of parameters, the receptive field, and computational complexity. This is followed by a discussion of techniques used to reduce the computational complexity and memory requirements of the neural networks used in this task. Finally, the techniques used by the winners of the latest speech enhancement challenges (DNS, Clarity) are shown and compared.

show abstract