Scalable-effort ConvNets for multilevel classification

Peluso, Valentino; Calimera, Andrea

doi:10.1145/3240765.3240845

Cited by 8 publications

(4 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, all objects belonging to the same category may not always have the same textures or colors, as shown in Modular Neural Networks for Low-Power Image Classification on Embedded Devices 1:7 more closely account for visual similarities to obtain significantly higher accuracy in classification without using textures and colors. [51][52][53][54] generate hierarchies using semantic similarity between categories or use pre-defined semantic hierarchies like WordNet [61]. WordNet is a large lexical database of the English language where nouns, verbs, adjectives, and adverbs are linked to one another using conceptual-semantic and lexical relations.…”

Section: Mnn-tree (Proposed Method)mentioning

confidence: 99%

See 1 more Smart Citation

Modular Neural Networks for Low-Power Image Classification on Embedded Devices

Goel

Aghajanzadeh

Tung

et al. 2020

ACM Trans. Des. Autom. Electron. Syst.

View full text Add to dashboard Cite

Embedded devices are generally small, battery-powered computers with limited hardware resources. It is difficult to run deep neural networks (DNNs) on these devices, because DNNs perform millions of operations and consume significant amounts of energy. Prior research has shown that a considerable number of a DNN's memory accesses and computation are redundant when performing tasks like image classification. To reduce this redundancy and thereby reduce the energy consumption of DNNs, we introduce the Modular Neural Network Tree architecture. Instead of using one large DNN for the classifier, this architecture uses multiple smaller DNNs (called modules) to progressively classify images into groups of categories based on a novel visual similarity metric. Once a group of categories is selected by a module, another module then continues to distinguish among the similar categories within the selected group. This process is repeated over multiple modules until we are left with a single category. The computation needed to distinguish dissimilar groups is avoided, thus reducing redundant operations, memory accesses, and energy. Experimental results using several image datasets reveal the effectiveness of our proposed solution to reduce memory requirements by 50% to 99%, inference time by 55% to 95%, energy consumption by 52% to 94%, and the number of operations by 15% to 99% when compared with existing DNN architectures, running on two different embedded systems: Raspberry Pi 3 and Raspberry Pi Zero. CCS Concepts: • Computing methodologies → Neural networks; Computer vision; • Computer systems organization → Embedded systems; • Hardware → Power and energy;

show abstract

Section: Mnn-tree (Proposed Method)mentioning

confidence: 99%

“…Images in the same category may have different colors and textures. Semantic similarity [51][52][53][54] Use semantic information from sources like WordNet to quantify the similarity.…”

Section: Techniquesmentioning

confidence: 99%

Modular Neural Networks for Low-Power Image Classification on Embedded Devices

Goel

Aghajanzadeh

Tung

et al. 2020

ACM Trans. Des. Autom. Electron. Syst.

View full text Add to dashboard Cite

show abstract

“…By performing pruning, quantization, and encoding, Deep Compression [4] reduces the model size by 95%. Pathlevel pruning is also seen in tree-based hierarchical DNNs [29,30]. Although these techniques can identify the unimportant connections, they create unwanted sparsity in DNNs.…”

Section: B Pruning Parameters and Connectionsmentioning

confidence: 99%

A Survey of Methods for Low-Power Deep Learning and Computer Vision

Goel¹,

Tung²,

Lu³

et al. 2020

2020 IEEE 6th World Forum on Internet of Things (WF-IoT)

View full text Add to dashboard Cite

Deep neural networks (DNNs) are successful in many computer vision tasks. However, the most accurate DNNs require millions of parameters and operations, making them energy, computation and memory intensive. This impedes the deployment of large DNNs in low-power devices with limited compute resources. Recent research improves DNN models by reducing the memory requirement, energy consumption, and number of operations without significantly decreasing the accuracy. This paper surveys the progress of low-power deep learning and computer vision, specifically in regards to inference, and discusses the methods for compacting and accelerating DNN models. The techniques can be divided into four major categories: (1) parameter quantization and pruning, (2) compressed convolutional filters and matrix factorization, (3) network architecture search, and (4) knowledge distillation. We analyze the accuracy, advantages, disadvantages, and potential solutions to the problems with the techniques in each category. We also discuss new evaluation metrics as a guideline for future research.

show abstract

“…Hybrid solutions may jointly exploit the complexity of the input problem with the accuracy imposed at the application level. For instance, the authors of [35] introduce the concept of multi-level classification where the classification task can be performed at different levels of semantic abstraction: the higher the abstraction, the easier the classification problem. Then, depending on the abstraction level and the desired accuracy, the ConvNet is tuned to achieve the maximum energy efficiency.…”

Section: Adaptive Convnetsmentioning

confidence: 99%

Energy-Accuracy Scalable Deep Convolutional Neural Networks: A Pareto Analysis

Peluso

Calimera

2019

VLSI-SoC: Design and Engineering of Electronics Systems Based on New Computing Paradigms

Self Cite

View full text Add to dashboard Cite

This work deals with the optimization of Deep Convolutional Neural Networks (ConvNets). It elaborates on the concept of Adaptive Energy-Accuracy Scaling through multi-precision arithmetic, a solution that allows ConvNets to be adapted at run-time and meet different energy budgets and accuracy constraints. The strategy is particularly suited for embedded applications made run at the "edge" on resourceconstrained platforms. After the very basics that distinguish the proposed adaptive strategy, the paper recalls the software-to-hardware vertical implementation of precision scalable arithmetic for ConvNets, then it focuses on the energy-driven per-layer precision assignment problem describing a meta-heuristic that searches for the most suited representation of both weights and activations of the neural network. The same heuristic is then used to explore the optimal trade-off providing the Pareto points in the energy-accuracy space. Experiments conducted on three different ConvNets deployed in real-life applications, i.e. Image Classification, Keyword Spotting, and Facial Expression Recognition, show adaptive ConvNets reach better energy-accuracy trade-off w.r.t. conventional static fixed-point quantization methods.

show abstract

Scalable-effort ConvNets for multilevel classification

Cited by 8 publications

References 7 publications

Modular Neural Networks for Low-Power Image Classification on Embedded Devices

Modular Neural Networks for Low-Power Image Classification on Embedded Devices

A Survey of Methods for Low-Power Deep Learning and Computer Vision

Energy-Accuracy Scalable Deep Convolutional Neural Networks: A Pareto Analysis

Contact Info

Product

Resources

About