Searching for Accurate Binary Neural Architectures

Shen, Mingzhu; Han, Kai; Xu, Chunjing; Wang, Yunhe

doi:10.1109/iccvw.2019.00256

Cited by 58 publications

(32 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Very recent work on binary NAS done concurrently with our work include [37] and [38]. As opposed to our work, [37] simple searches for the number of channels in each layer inside a ResNet. [38] uses a completely different search space and training strategy.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

BATS: Binary ArchitecTure Search

Bulat

Martínez

Tzimiropoulos

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper proposes Binary ArchitecTure Search (BATS), a framework that drastically reduces the accuracy gap between binary neural networks and their real-valued counterparts by means of Neural Architecture Search (NAS). We show that directly applying NAS to the binary domain provides very poor results. To alleviate this, we describe, to our knowledge, for the first time, the 3 key ingredients for successfully applying NAS to the binary domain. Specifically, we (1) introduce and design a novel binary-oriented search space, (2) propose a new mechanism for controlling and stabilising the resulting searched topologies, (3) propose and validate a series of new search strategies for binary networks that lead to faster convergence and lower search times. Experimental results demonstrate the effectiveness of the proposed approach and the necessity of searching in the binary space directly. Moreover, (4) we set a new state-of-the-art for binary neural networks on CIFAR10, CIFAR100 and ImageNet datasets. Code will be made available.

show abstract

Section: Related Workmentioning

confidence: 99%

“…On ImageNet, using 1.55 × 10 8 FLOPs, we obtain a Top-1 accuracy of 66.1% vs 58.76% using 1.63 × 10 8 FLOPS in [38]. On CIFAR-10 we score a top-1 accuracy of 96.1% vs 94.43% in [37]. While [37] achieves an accuracy of 69.65% on ImageNet, they use 6.6 × 10 8 FLOPS (4.2× more than our biggest model).…”

Section: Related Workmentioning

confidence: 99%

BATS: Binary ArchitecTure Search

Bulat

Martínez

Tzimiropoulos

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Shen et al [37] encode the number of channels in each layer into search space and optimize using the evolutionary algorithm. This approach can identify binary neural architectures for obtaining high precision with as few computation costs as possible.…”

Section: F Neural Architecture Searchmentioning

confidence: 99%

A Review of Recent Advances of Binary Neural Networks for Edge Computing

Zhao

Gong

et al. 2021

IEEE J. Miniat. Air Space Syst.

View full text Add to dashboard Cite

Edge computing is promising to become one of the next hottest topics in artificial intelligence because it benefits various evolving domains such as real-time unmanned aerial systems, industrial applications, and the demand for privacy protection. This paper reviews recent advances on binary neural network (BNN) and 1-bit CNN technologies that are well suitable for front-end, edge-based computing. We introduce and summarize existing work and classify them based on gradient approximation, quantization, architecture, loss functions, optimization method, and binary neural architecture search. We also introduce applications in the areas of computer vision and speech recognition and discuss future applications for edge computing.

show abstract

“…However, the accuracy of QNNs is still worse than the full-precision versions. The second approach improves the performance of QNNs by expanding the channel number of the convolutional layers [17,18]. Shen et al [18] applied this idea to 1-bit QNNs by encoding the channel expansion rates of all convolutional layers and utilizing the evolutionary algorithms (EAs) to search for the expansion policy.…”

Section: Introductionmentioning

confidence: 99%

“…The second approach improves the performance of QNNs by expanding the channel number of the convolutional layers [17,18]. Shen et al [18] applied this idea to 1-bit QNNs by encoding the channel expansion rates of all convolutional layers and utilizing the evolutionary algorithms (EAs) to search for the expansion policy. Experiments show that their method can build QNNs with acceptable parameters increment and the accuracy close to the full-precision counterparts.…”

Section: Introductionmentioning

confidence: 99%

Evolving Quantized Neural Networks for Image Classification Using A Multi-Objective Genetic Algorithm

Wang

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Recently, many model quantization approaches have been investigated to reduce the model size and improve the inference speed of convolutional neural networks (CNNs). However, these approaches usually inevitably lead to a decrease in classification accuracy. To address this problem, this paper proposes a mixed precision quantization method combined with channel expansion of CNNs by using a multi-objective genetic algorithm, called MOGAQNN. In MOGAQNN, each individual in the population is used to encode a mixed precision quantization policy and a channel expansion policy. During the evolution process, the two polices are optimized simultaneously by the non-dominated sorting genetic algorithm II (NSGA-II). Finally, we choose the best individual in the last population and evaluate its performance on the test set as the final performance. The experimental results of five popular CNNs on two benchmark datasets demonstrate that MOGAQNN can greatly reduce the model size and improve the classification accuracy at the same time.

show abstract

Searching for Accurate Binary Neural Architectures

Cited by 58 publications

References 16 publications

BATS: Binary ArchitecTure Search

BATS: Binary ArchitecTure Search

A Review of Recent Advances of Binary Neural Networks for Edge Computing

Evolving Quantized Neural Networks for Image Classification Using A Multi-Objective Genetic Algorithm

Contact Info

Product

Resources

About