How Initialization is Related to Deep Neural Networks Generalization Capability: Experimental Study

Sandjakoska, Ljubinka; Stojanovska, Frosina

doi:10.1109/icest49890.2020.9232882

Cited by 8 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In Sandjakoska and Stojanovska [20] the authors present a similar experimental study to ours, but with different DNNs and the QM9 dataset, that provides quantum chemical properties for a relevant, consistent, and comprehensive chemical space of small organic molecules. It is interesting how they consider initialization as a junction point between optimization and regularization.…”

Section: Related Workmentioning

confidence: 87%

Deep Image Prior for medical image denoising, a study about parameter initialization

Sapienza

Franchini

Govi

et al. 2022

Front. Appl. Math. Stat.

View full text Add to dashboard Cite

Convolutional Neural Networks are widely known and used architectures in image processing contexts, in particular for medical images. These Deep Learning techniques, known for their ability to extract high-level features, almost always require a labeled dataset, a process that can be computationally expensive. Most of the time in the biomedical context, when images are used they are noisy and the ground-truth is unknown. For this reason, and in the context of Green Artificial Intelligence, recently, an unsupervised method that employs Convolutional Neural Networks, or more precisely autoencoders, has appeared in the panorama of Deep Learning. This technique, called Deep Image Prior (DIP) by the authors, can be used in areas such as denoising, superresolution, and inpainting. Starting from these assumptions, this work analyses the robustness of these networks with respect to different types of initialization. First of all, we analyze the different types of parameters: related to the Batch Norm and the Convolutional layers. For the results, we focus on the speed of convergence and the maximum performance obtained. However, this paper aims to apply acquired information on Computer Tomography noised images. In fact, the final purpose is to test the best initializations of the first phase on a phantom image and then on a real Computer Tomography one. In fact, Computer Tomography together with Magnetic Resonance Imaging and Positron Emission Tomography are some of the diagnostic tools currently available to neuroscientists and oncologists. This work shows how initializations affect final performances and, in addition, how they should be used in the medical image reconstruction field. The section on numerical experiments shows results that on the one hand confirm the importance of a good initialization to obtain fast convergence and high performance; on the other hand, it shows how the method is robust to the processing of different image types: natural and medical. Not a single good initialization is discovered, but many of them could be chosen, according to specific necessities of the single problem.

show abstract

Section: Related Workmentioning

confidence: 87%

Deep Image Prior for medical image denoising, a study about parameter initialization

Sapienza

Franchini

Govi

et al. 2022

Front. Appl. Math. Stat.

View full text Add to dashboard Cite

show abstract

“…El 60 % de las imágenes se consideraron como datos de entrenamiento y el 20 % de validación, mientras que el 20 % restante, se incluye en los datos de prueba del modelo (ver Tabla 2), es decir, que este es entrenado y validado paralelamente para finalmente ser probado con imágenes nuevas que no han sido consideradas con anterioridad. En [39] se indica que los modelos que generalizan adecuadamente muestran métricas de exactitud y pérdida similares en el entrenamiento y validación, evitándose el sobreajuste.…”

Section: Set De Datosunclassified

Transfer learning en la clasificación binaria de imágenes térmicas

2021

View full text Add to dashboard Cite

La clasificación de imágenes térmicas es un aspecto clave en el sector industrial, debido a que suele ser el punto de partida en la detección de fallos en equipos eléctricos. En algunos casos, esta tarea se automatiza mediante el uso de técnicas tradicionales de inteligencia artificial, mientras que en otros, es realizada de manera manual, lo cual puede traer consigo altas tasas de error humano. Este artículo presenta un análisis comparativo entre once arquitecturas de transfer learning (AlexNet, VGG16, VGG19, ResNet, DenseNet, MobileNet v2, GoogLeNet, ResNeXt, Wide ResNet, MNASNet y ShuffleNet) mediante el uso de fine-tuning, con la finalidad de realizar una clasificación binaria de imágenes térmicas en una red de distribución eléctrica. Para ello, se dispone de una base de datos con 815 imágenes, divididas mediante la técnica tipo hold-out 60-20-20 y validación cruzada con 5-folds, para finalmente analizar su rendimiento mediante el test de Friedman. Luego de los experimentos, se obtuvieron resultados satisfactorios con exactitudes superiores a 85 % en diez de las arquitecturas previamente entrenadas. Sin embargo, la arquitectura que no se entrenó previamente presentó una exactitud baja; concluyéndose que la aplicación de transfer learning mediante el uso de arquitecturas previamente entrenadas es un mecanismo adecuado en la clasificación de este tipo de imágenes, y representa una alternativa confiable frente a técnicas tradicionales de inteligencia artificial.

show abstract

“…Recently, deep learning has recently received great research efforts in various fields and applications, including fault detection [21]. The deep learning model is a hierarchical learning structure, in which complex nonlinear functions are used [22]. In an attempt to detect faults in a PV system based on deep learning, aerial images obtained from drones were used.…”

Section: Introductionmentioning

confidence: 99%

IoT-based monitoring and shading faults detection for a PV water pumping system using deep learning approach

Qasim Obaidi,

Derbel

2023

Bulletin EEI

View full text Add to dashboard Cite

One of the major challenges facing photovoltaic (PV) systems is fault detection. Artificial intelligence (AI) is one of the main popular techniques used in error detection due to its ability to extract signal and image features. In this paper, a deep learning approach based on convolutional neural network (CNN) and internet of things (IoT) technology are used to detect and locate shading faults for a PV water pumping system. The current and voltage signals generated by the PV panels as well as temperature and radiation were used to convert them into 3D images and then upload to a deep learning algorithm. The PV system and fault detection algorithms were simulated by MATLAB. The obtained results indicate that the performance of the proposed deep learning approach to detect and locate faults is better than the traditional statistical methods and other machine learning methods.

show abstract

How Initialization is Related to Deep Neural Networks Generalization Capability: Experimental Study

Cited by 8 publications

References 25 publications

Deep Image Prior for medical image denoising, a study about parameter initialization

Deep Image Prior for medical image denoising, a study about parameter initialization

Transfer learning en la clasificación binaria de imágenes térmicas

IoT-based monitoring and shading faults detection for a PV water pumping system using deep learning approach

Contact Info

Product

Resources

About