BinaryDenseNet: Developing an Architecture for Binary Neural Networks

Bethge, Joseph; Yang, Haojin; Bornstein, Marvin; Meinel, Christoph

doi:10.1109/iccvw.2019.00244

Cited by 50 publications

(59 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The grouped convolutions are applied for adopting the binarized depth-wise separable convolution in He et al (2018) and Phan et al (2020). In Bethge et al (2019Bethge et al ( , 2020, shortcut is concatenated to expand channels in dense neural networks.…”

Section: Binarized Cnnsmentioning

confidence: 99%

“…Despite many benefits above, the low precision of the binarized operations in BNNs degrades the classification ability on modern CNNs, thus limiting their applications. Improved BNN structures have been developed for reducing the gap of the classification accuracy degraded over real-valued CNNs ( Lin, Zhao & Pan, 2017 ; Liang et al, 2018 ; Liu et al, 2018 ; He et al, 2018 ; Zhuang et al, 2019 ; Shen et al, 2019 ; Chakraborty et al, 2019 ; Bethge et al, 2019 ; Phan et al, 2020 ; Bethge et al, 2020 ; Liu et al, 2020 ). Besides, new training methods and optimizing tricks for BNNs have been researched for obtaining better classification accuracy ( Alizadeh et al, 2018 ; Bulat & Tzimiropoulos, 2019 ; Zhu, Dong & Su, 2019 ; Wang et al, 2019 ; Hubara et al, 2017 ; Ghasemzadeh, Samragh & Koushanfar, 2018 ; Gu et al, 2019 ; Helwegen et al, 2019 ; Ding et al, 2019 ; Martinez et al, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

AresB-Net: accurate residual binarized neural networks using shortcut concatenation and shuffled grouped convolution

Kim

2021

PeerJ Computer Science

View full text Add to dashboard Cite

This article proposes a novel network model to achieve better accurate residual binarized convolutional neural networks (CNNs), denoted as AresB-Net. Even though residual CNNs enhance the classification accuracy of binarized neural networks with increasing feature resolution, the degraded classification accuracy is still the primary concern compared with real-valued residual CNNs. AresB-Net consists of novel basic blocks to amortize the severe error from the binarization, suggesting a well-balanced pyramid structure without downsampling convolution. In each basic block, the shortcut is added to the convolution output and then concatenated, and then the expanded channels are shuffled for the next grouped convolution. In the downsampling when stride >1, our model adopts only the max-pooling layer for generating low-cost shortcut. This structure facilitates the feature reuse from the previous layers, thus alleviating the error from the binarized convolution and increasing the classification accuracy with reduced computational costs and small weight storage requirements. Despite low hardware costs from the binarized computations, the proposed model achieves remarkable classification accuracies on the CIFAR and ImageNet datasets.

show abstract

Section: Binarized Cnnsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

AresB-Net: accurate residual binarized neural networks using shortcut concatenation and shuffled grouped convolution

Kim

2021

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…The grouped convolutions are applied for adopting the binarized depth-wise separable convolution in He et al (2018); Phan et al (2020). In Bethge et al (2019Bethge et al ( , 2020, shortcut is concatenated to expand channels in dense neural networks.…”

Section: Binarized Convolutional Neural Networkmentioning

confidence: 99%

“…Sci. reviewing PDF | (CS- 2020:11:55497:1:1:NEW 8 Feb 2021) Manuscript to be reviewed Computer Science Chakraborty et al, 2019;Bethge et al, 2019;Phan et al, 2020;Bethge et al, 2020;Liu et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

“…However, it is concerned that the stacking structure of residual convolution layers accumulates errors from each binarized layer, which can limit the performance of residual CNNs. The feature reuse of BNNs in Bethge et al (2019Bethge et al ( , 2020 concatenates shortcuts to expand output channel, making features from the shortcut pass to the next block. These existing methods in Liu et al (2018); Bethge et al (2019) adopt point-wise convolutions in the channel expansion, which can increase computational costs in BNNs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Peer Review #1 of "AresB-Net: accurate residual binarized neural networks using shortcut concatenation and shuffled grouped convolution (v0.1)"

2021

View full text Add to dashboard Cite

This paper proposes a novel network model to achieve better accurate residual binarized convolutional neural networks (CNNs), denoted as AresB-Net. Even though residual CNNs enhance the classification accuracy of binarized neural networks (BNNs) with increasing feature resolution, the degraded classification accuracy is still the primary concern compared with real-valued residual CNNs. AresB-Net consists of novel basic blocks to amortize the severe error from the binarization, suggesting a well-balanced pyramid structure without downsampling convolution. In each basic block, the shortcut is added to the convolution output and then concatenated, and then the expanded channels are shuffled for the next grouped convolution. In the downsampling when stride > 1, our model adopts only the max-pooling layer for generating low-cost shortcut. This structure facilitates the feature reuse from the previous layers, thus alleviating the error from the binarized convolution and increasing the classification accuracy with reduced computational costs and small weight storage requirements. Despite low hardware costs from the binarized computations, the proposed model achieves remarkable classification accuracies on the CIFAR and ImageNet datasets.

show abstract

Squeeze-and-Threshold Based Quantization for Low-Precision Neural Networks

Waschneck

Mayr³

2021

Proceedings of the International Neural Networks Society

View full text Add to dashboard Cite

BinaryDenseNet: Developing an Architecture for Binary Neural Networks

Cited by 50 publications

References 15 publications

AresB-Net: accurate residual binarized neural networks using shortcut concatenation and shuffled grouped convolution

AresB-Net: accurate residual binarized neural networks using shortcut concatenation and shuffled grouped convolution

Peer Review #1 of "AresB-Net: accurate residual binarized neural networks using shortcut concatenation and shuffled grouped convolution (v0.1)"

Squeeze-and-Threshold Based Quantization for Low-Precision Neural Networks

Contact Info

Product

Resources

About