DeepQGHO: Quantized Greedy Hyperparameter Optimization in Deep Neural Networks for on-the-Fly Learning

Chowdhury, Anjir Ahmed; Hossen, Md Abir; Azam, Md Ali; Rahman, Md. Hafizur

doi:10.1109/access.2022.3141781

Cited by 12 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Anjir A. Chowdhury et al concentrated on the role of hyper-parameter optimization in the performance and reliability of deep learning outcomes [33]. They compared several HPO algorithms to obtain better validation accuracy in DNNs and concluded that most of them are computationally expensive.…”

Section: Hyper-parameter Tuning In MLmentioning

confidence: 99%

Hyper-Parameter Optimization of Stacked Asymmetric Auto-Encoders for Automatic Personality Traits Perception

Zaferani

Teshnehlab

Khodadadian

et al. 2022

Sensors

View full text Add to dashboard Cite

In this work, a method for automatic hyper-parameter tuning of the stacked asymmetric auto-encoder is proposed. In previous work, the deep learning ability to extract personality perception from speech was shown, but hyper-parameter tuning was attained by trial-and-error, which is time-consuming and requires machine learning knowledge. Therefore, obtaining hyper-parameter values is challenging and places limits on deep learning usage. To address this challenge, researchers have applied optimization methods. Although there were successes, the search space is very large due to the large number of deep learning hyper-parameters, which increases the probability of getting stuck in local optima. Researchers have also focused on improving global optimization methods. In this regard, we suggest a novel global optimization method based on the cultural algorithm, multi-island and the concept of parallelism to search this large space smartly. At first, we evaluated our method on three well-known optimization benchmarks and compared the results with recently published papers. Results indicate that the convergence of the proposed method speeds up due to the ability to escape from local optima, and the precision of the results improves dramatically. Afterward, we applied our method to optimize five hyper-parameters of an asymmetric auto-encoder for automatic personality perception. Since inappropriate hyper-parameters lead the network to over-fitting and under-fitting, we used a novel cost function to prevent over-fitting and under-fitting. As observed, the unweighted average recall (accuracy) was improved by 6.52% (9.54%) compared to our previous work and had remarkable outcomes compared to other published personality perception works.

show abstract

Section: Hyper-parameter Tuning In MLmentioning

confidence: 99%

Hyper-Parameter Optimization of Stacked Asymmetric Auto-Encoders for Automatic Personality Traits Perception

Zaferani

Teshnehlab

Khodadadian

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…For instance, Model design techniques emphasize on designing models with a reduced number of parameters without compromising accuracy, thereby enabling it to fit and execute within the available IoT device memory [10]. Also, Model compression techniques such as quantization [11] and pruning [10] can be used. Where quantization takes out the expensive floating-point operations by reducing it to a Q-bit fixed-point number, and pruning removes the unnecessary connections between the model layers.…”

Section: A Machine Learning On Microcontrollersmentioning

confidence: 99%

Multi-Component Optimization and Efficient Deployment of Neural-Networks on Resource-Constrained IoT Hardware

Sudharsan¹,

Sundaram²,

Patel³

et al. 2022

Preprint

View full text Add to dashboard Cite

The majority of IoT devices like smartwatches, smart plugs, HVAC controllers, etc., are powered by hardware with a constrained specification (low memory, clock speed and processor) which is insufficient to accommodate and execute large, high-quality models. On such resource-constrained devices, manufacturers still manage to provide attractive functionalities (to boost sales) by following the traditional approach of programming IoT devices/products to collect and transmit data (image, audio, sensor readings, etc.) to their cloud-based ML analytics platforms. For decades, this online approach has been facing issues such as compromised data streams, non-real-time analytics due to latency, bandwidth constraints, costly subscriptions, recent privacy issues raised by users and the GDPR guidelines, etc. In this paper, to enable ultra-fast and accurate AI-based offline analytics on resource-constrained IoT devices, we present an end-toend multi-component model optimization sequence and open-source its implementation. Researchers and developers can use our optimization sequence to optimize high memory, computation demanding models in multiple aspects in order to produce small size, low latency, low-power consuming models that can comfortably fit and execute on resource-constrained hardware. The experimental results show that our optimization components can produce models that are; (i) 12.06 x times compressed; (ii) 0.13% to 0.27% more accurate; (iii) Orders of magnitude faster unit inference at 0.06 ms. Our optimization sequence is generic and can be applied to any state-of-the-art models trained for anomaly detection, predictive maintenance, robotics, voice recognition, and machine vision.

show abstract

“…These geometric features are frequently extracted and measured using deep learning techniques or traditional image processing techniques, which are labor-intensive, prone to errors, and have low rates of efficiency. In order to efficiently automate this process [9] [10] [11], our proposed algorithm makes use of computer vision and deep learning techniques to automatically extract each cell's geometric properties from images of biofilms.…”

Section: Introductionmentioning

confidence: 99%

BiofilmScanner: A Computational Intelligence Approach to Obtain Bacterial Cell Morphological Attributes from Biofilm Image

Rahman¹,

Azam²,

Hossen³

et al. 2023

Preprint

View full text Add to dashboard Cite

Desulfovibrio alaskensis G20 (DA-G20) is utilized as a model for sulfate-reducing bacteria (SRB) that are associated with corrosion issues caused by microorganisms. SRB-based biofilms are thought to be responsible for the billion-dollar-per-year bio-corrosion of metal infrastructure. Understanding the extraction of the bacterial cells' shape and size properties in the SRB-biofilm at different growth stages will assist with the design of anti-corrosion techniques. However, numerous issues affect current approaches, including time-consuming geometric property extraction, low efficiency, and high error rates. This paper proposes BiofilScanner, a Yolact-based deep learning method integrated with invariant moments to address these problems. Our approach efficiently detects and segments bacterial cells in an SRB image while simultaneously invariant moments measure the geometric characteristics of the segmented cells with low errors. The numerical experiments of the proposed method demonstrate that the BiofilmScanner is 2.1x and 6.8x faster than our earlier Mask-RCNN and DLv3+ methods for detecting, segmenting, and measuring the geometric properties of the cell. Furthermore, the BiofilmScanner achieved an F1-score of 85.28% while Mask-RCNN and DLv3+ obtained F1-scores of 77.67% and 75.18%, respectively.

show abstract

DeepQGHO: Quantized Greedy Hyperparameter Optimization in Deep Neural Networks for on-the-Fly Learning

Cited by 12 publications

References 29 publications

Hyper-Parameter Optimization of Stacked Asymmetric Auto-Encoders for Automatic Personality Traits Perception

Hyper-Parameter Optimization of Stacked Asymmetric Auto-Encoders for Automatic Personality Traits Perception

Multi-Component Optimization and Efficient Deployment of Neural-Networks on Resource-Constrained IoT Hardware

BiofilmScanner: A Computational Intelligence Approach to Obtain Bacterial Cell Morphological Attributes from Biofilm Image

Contact Info

Product

Resources

About