MeliusNet: An Improved Network Architecture for Binary Neural Networks

Bethge, Joseph; Bartz, Christian; Yang, Haojin; Chen, Ying; Meinel, Christoph

doi:10.1109/wacv48630.2021.00148

Cited by 31 publications

(12 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We study the performance of DIR-Net over ResNet-18, ResNet-34, MobileNetV1, DARTS, and EfficientNet-B0 structures on the large-scale ImageNet dataset. Table 6 lists the comparison with several SOTA quantization methods, including BWN [68], HWGQ [50], TWN [48], LQ-Net [87], DoReFa-Net [90], ABC-Net [51], Bi-Real [57], XNOR++ [6], BWHN [36], SQ-BWN and SQ-TWN [19], PCNN [26], BONN [27], Si-BNN [76], Real-to-Bin [59], MeliusNet [5], and ReActNet [56].…”

Section: Imagenetmentioning

confidence: 99%

“…6(b)). Fourth, we compare proposed DIR-Net with more SOTA binarization approaches in Table 6 (BONN [27], Si-BNN [76], PCNN [26], Real-to-Bin [59], MeliusNet [5], and ReActNet [56]) and evaluate it on compact networks (EfficientNet [73], MobileNet [34] and DARTS [54]). The results show that our DIR-Net is versatile and effective, and can improve the performance on these structures.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Forward and Backward Information Retention for Accurate Binary Neural Networks

Qin

Gong

Liu

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

270

234

View full text Add to dashboard Cite

Model binarization is an effective method of compressing neural networks and accelerating their inference process, which enables state-of-the-art models to run on resource-limited devices. Recently, advanced binarization methods have been greatly improved by minimizing the quantization error directly in the forward process. However, a significant performance gap still exists between the 1-bit model and the 32-bit one. The empirical study shows that binarization causes a great loss of information in the forward and backward propagation which harms the performance of binary neural networks (BNNs), and the limited information representation ability of binarized parameter is one of the bottlenecks of BNN performance. We present a novel Distributionsensitive Information Retention Network (DIR-Net) to retain the information of the forward activations and backward gradients, which improves BNNs

show abstract

Section: Imagenetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Forward and Backward Information Retention for Accurate Binary Neural Networks

Qin

Gong

Liu

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

270

234

View full text Add to dashboard Cite

show abstract

“…Since the proposal of BNNs [1], [2], there have been significant efforts committed to closing the accuracy gap between BNNs and the full-precision DNNs. These include adding learnable scaling factors (also known as gain term) [33]- [36], adopting multiple basis [33]- [35], [37], designing BNN-oriented network structure [16], [18], [23], [38], and reforming BNN training methodologies [22], [23], [36], [39]- [41]. More detailed information can be found in the BNN surveys [42], [43].…”

Section: A Binary Neural Networkmentioning

confidence: 99%

“…Though BNNs significantly alleviate the computation cost, they suffer from non-trivial performance degradation largely due to the imperfection of the binarization function and its derivative approximation [15], [16]. Most existing BNN works binarize the inputs and weights into binary (+1/-1) tensors through a deterministic sign function.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

GAAF: Searching Activation Functions for Binary Neural Networks through Genetic Algorithm

Li¹,

Geng²,

Stein³

et al. 2022

Preprint

View full text Add to dashboard Cite

Binary neural networks (BNNs) show promising utilization in cost and power-restricted domains such as edge devices and mobile systems. This is due to its significantly less computation and storage demand, but at the cost of degraded performance. To close the accuracy gap, in this paper we propose to add a complementary activation function (AF) ahead of the sign based binarization, and rely on the genetic algorithm (GA) to automatically search for the ideal AFs. These AFs can help extract extra information from the input data in the forward pass, while allowing improved gradient approximation in the backward pass. Fifteen novel AFs are identified through our GAbased search, while most of them show improved performance (up to 2.54% on ImageNet) when testing on different datasets and network models. Our method offers a novel approach for designing general and application-specific BNN architecture. Our code is available at http://github.com/flying-Yan/GAAF.

show abstract

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

Chen

Ren

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

MeliusNet: An Improved Network Architecture for Binary Neural Networks

Cited by 31 publications

References 20 publications

Forward and Backward Information Retention for Accurate Binary Neural Networks

Forward and Backward Information Retention for Accurate Binary Neural Networks

GAAF: Searching Activation Functions for Binary Neural Networks through Genetic Algorithm

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

Contact Info

Product

Resources

About