Should You Go Deeper? Optimizing Convolutional Neural Network Architectures without Training

Richter, Mats L.; Schöning, Julius; Wiedenroth, Anna; Krumnack, Ulf

doi:10.1109/icmla52953.2021.00159

Cited by 5 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, deeper ANN architectures amplify hardware requirements and demand larger datasets for training. When considering AI-based CLCS in embedded devices, the question arises: Should one continue to pursue deeper architectures [6,7] In AI applications where functional safety and trustworthiness are not critical requirements, a noticeable trend emerges when examining the ANN architectures developed: predictive performance improves linearly, while the number of trainable parameters and, thus, computational complexity increase exponentially [8].…”

Section: Pipeline Of Applying Aimentioning

confidence: 99%

“…It is important to carefully consider the design of the AI-based controller to minimize the number of trainable weights, reducing complexity and improving functional safety. Tailoring the ANN based on specific needs should also be considered in this design [6,7].…”

Section: Functional Safety For Ai-empowered and -Based Closed-loop Co...mentioning

confidence: 99%

“…Thus, research should focus on tiny ANNs, i.e., with few trainable parameters. As one possible option to create tiny ANNs, the elimination of unproductive layers of an ANN is already feasible [6,7].…”

Section: Research Direction: Tiny Aimentioning

confidence: 99%

See 2 more Smart Citations

Safe and Trustful AI for Closed-Loop Control Systems

Schöning,

Pfisterer

2023

Electronics

View full text Add to dashboard Cite

In modern times, closed-loop control systems (CLCSs) play a prominent role in a wide application range, from production machinery via automated vehicles to robots. CLCSs actively manipulate the actual values of a process to match predetermined setpoints, typically in real time and with remarkable precision. However, the development, modeling, tuning, and optimization of CLCSs barely exploit the potential of artificial intelligence (AI). This paper explores novel opportunities and research directions in CLCS engineering, presenting potential designs and methodologies incorporating AI. Combining these opportunities and directions makes it evident that employing AI in developing and implementing CLCSs is indeed feasible. Integrating AI into CLCS development or AI directly within CLCSs can lead to a significant improvement in stakeholder confidence. Integrating AI in CLCSs raises the question: How can AI in CLCSs be trusted so that its promising capabilities can be used safely? One does not trust AI in CLCSs due to its unknowable nature caused by its extensive set of parameters that defy complete testing. Consequently, developers working on AI-based CLCSs must be able to rate the impact of the trainable parameters on the system accurately. By following this path, this paper highlights two key aspects as essential research directions towards safe AI-based CLCSs: (I) the identification and elimination of unproductive layers in artificial neural networks (ANNs) for reducing the number of trainable parameters without influencing the overall outcome, and (II) the utilization of the solution space of an ANN to define the safety-critical scenarios of an AI-based CLCS.

show abstract

Section: Pipeline Of Applying Aimentioning

confidence: 99%

Section: Functional Safety For Ai-empowered and -Based Closed-loop Co...mentioning

confidence: 99%

See 1 more Smart Citation

Safe and Trustful AI for Closed-Loop Control Systems

Schöning,

Pfisterer

2023

Electronics

View full text Add to dashboard Cite

show abstract

“…Thus, CNN automatically learns highlevel features to discriminate between different objects. Currently, CNN architectures can be divided into two main types [39]:…”

Section: Cnn Architectures and Transfer Learningmentioning

confidence: 99%

A Huanglongbing Detection Method for Orange Trees Based on Deep Neural Networks and Transfer Learning

2022

View full text Add to dashboard Cite

Huanglongbing (HLB) is one of the most threatening diseases for citrus production and it has caused significant economic damage worldwide. Hence, computer-vision systems that are based on convolutional neural networks (CNNs) can detect HLB accurately. Moreover, the detection system should be able to discriminate between HLB and other citrus abnormalities to ensure that any treatments are effective. Besides, the causal pathogen of HLB is usually detected and diagnosed by the quantitative realtime polymerase chain reaction (qPCR) test, which is costly. Consequently, it is difficult to collect large datasets to train CNN-based systems. In this case, transfer learning from pre-trained CNNs is a solution for building an HLB-detection system using small-sized datasets. This paper evaluates two kinds of CNN architectures: series network (represented by AlexNet, VGG16, and VGG19 models) and directed acyclic graph (DAG) network (represented by ResNet18, GoogLeNet, and Inception-V3 models). These pre-trained CNNs are fine-tuned to distinguish HLB, healthy cases, and 10 kinds of abnormalities of the Citrus sinensis species, which is commonly known as sweet orange. The dataset includes 953 color images, where the leaf samples were collected from orange groves in north Mexico. The 10-fold cross-validation results show that all the CNNs present a 95% or higher HLB sensitivity. However, the number of trainable parameters impacts HLB detection more than the network's depth. Specifically, VGG19, with 19 layers and 144 M parameters, reached a perfect sensitivity for all cross-validation experiments; whereas Inception-V3, with 48 layers and 24 M parameters, reached 95% sensitivity to HLB detection. This outcome happens because a higher number of parameters compensates for the limited number of HLB cases, so VGG19 can successfully transfer the learned characteristics to new cases. This study gives guidance when choosing an adequate CNN to efficiently detect HLB and other orange abnormalities. Besides, a detection scheme is proposed to be further implemented in a portable system to detect HLB in situ, potentially helping to reduce economic losses for small growers from low-income regions. INDEX TERMSHuanglongbing detection, Citrus sinensis orange, Convolutional neural networks, Transfer learning, Hand-crafted features. America, Asia, and South Africa. HLB's visible symptoms include blotchy mottle leaves, stunted growth, yellow shoot, reduced fruit size, corky veins, root decline, and dieback [3]. The suspicious causal pathogen of this disease is a bacterium of the genus Candidatus Liberibacter. Three bacterial species are known to cause HLB: Ca. L. asiaticus, Ca. L. africanus, and Ca. L. americanus. Transmission of the bacteria occurs through an insect vector. Both Ca. L. asiaticus and Ca. L. americanus are transmitted by the Asian citrus psyllid Diaphorina citri, whereas Ca. L. africanus is transmitted by

show abstract