Quantum-Inspired Neural Architecture Search

Evolving Deep DenseBlock Architecture Ensembles for Image Classification

“…Furthermore, the proposed SODBAE (BLSK-NS) model achieves an error rate of 9.72% and outperforms the remaining related studies, including Q-NAS [20], EPSO-CNN [8], SMAC with predictive termination [9], SMAC [9], HORD [10], NMM [11], PSO-b [34], and GA-CNN [35], by 1.28%, 10.13%, 7.48%, 7.75%, 10.82%, 0.33%, 8.81% and 15.69%, respectively. Moreover, the proposed SODBAE (BLSK-S) method achieves an error rate of 13.74% and outperforms related studies, e.g., EPSO-CNN [8], SMAC with predictive termination [9], SMAC [9], HORD [10], PSO-b [34], and GA-CNN [35], by 6.11%, 3.46%, 3.73%, 6.8%, 4.79% and 11.67%, respectively.…”

“…Large-Scale Evolution [1], Genetic CNN [2], and Hierarchical Representations [4] achieved comparatively better classification performances, but at the expense of significantly larger computational costs, i.e., 250 × 264, 20 × 24 and 200 × 36 GPU-hours. Q-NAS [20] illustrated a less competitive performance, but also with a significantly large computational cost, i.e., 20 × 50 GPU-hours. As such, these experimental settings may not be accessible to average consumers, owing to the vast resources requirements.…”

“…Deep architecture generation is a challenging task, since related studies [1,2,4,20] tend to create search spaces that are immense and feasibly impossible to properly navigate without a huge amount of resources for parallelisation. In this research, we approach the problem differently by proposing novel weight inheritance schemes, a DenseBlock skeleton architecture, as well as an adaptive cosine oriented PSO search mechanism to balance between devising versatile architectures and maintaining optimal training costs.…”

“…In comparison with these studies, our proposed models, i.e., SODBAE with BLSK-S and SODBAE with BLSK-NS, have the costs of 100 and 150 GPU-hours, respectively, and illustrate better trade-offs between performance and computational efficiency. Specifically, the proposed SODBAE (BLSK-NS) model achieves an error rate of 9.72% and it has a proportion (i.e., 0.2273%, 31.25%, 2.0833% and 15%) of the computational costs of Large-Scale Evolution [1], Genetic CNN [2], Hierarchical Representations [4], and Q-NAS [20], respectively. Additionally, the proposed SODBAE (BLSK-S) model achieves an error rate of 13.74% and it has a proportion (i.e., 0.152%, 20.83%, 1.389% and 10%) of the computational costs of Large-Scale Evolution [1], Genetic CNN [2], Hierarchical Representations [4], and Q-NAS [20], respectively.…”

“…Particularly, one key method for comparison is the Q-NAS system [20]. This work used a quantum-inspired evolutionary search method in order to generate architectures and evaluate them on the CIFAR-10 dataset, achieving an error rate of 11.00% following 50 h of optimisation using 20 GPUs (1000 GPU-hours).…”