Post-synaptic Potential Regularization Has Potential

Tartaglione, Enzo; Perlo, Daniele; Grangetto, Marco

doi:10.1007/978-3-030-30484-3_16

Cited by 6 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… Of course many other issues have to be taken into account at training time, like the use of a validation-set to tune the hyper-parameters, using a good regularization policy etc. but these very general issues have been exhaustively discussed in many other works [ 32 , 33 , 34 ]. …”

Section: Resultsmentioning

confidence: 99%

Unveiling COVID-19 from CHEST X-Ray with Deep Learning: A Hurdles Race with Small Data

Tartaglione

Barbano

Berzovini

et al. 2020

IJERPH

Self Cite

181

122

View full text Add to dashboard Cite

The possibility to use widespread and simple chest X-ray (CXR) imaging for early screening of COVID-19 patients is attracting much interest from both the clinical and the AI community. In this study we provide insights and also raise warnings on what is reasonable to expect by applying deep learning to COVID classification of CXR images. We provide a methodological guide and critical reading of an extensive set of statistical results that can be obtained using currently available datasets. In particular, we take the challenge posed by current small size COVID data and show how significant can be the bias introduced by transfer-learning using larger public non-COVID CXR datasets. We also contribute by providing results on a medium size COVID CXR dataset, just collected by one of the major emergency hospitals in Northern Italy during the peak of the COVID pandemic. These novel data allow us to contribute to validate the generalization capacity of preliminary results circulating in the scientific community. Our conclusions shed some light into the possibility to effectively discriminate COVID using CXR.

show abstract

Section: Resultsmentioning

confidence: 99%

Unveiling COVID-19 from CHEST X-Ray with Deep Learning: A Hurdles Race with Small Data

Tartaglione

Barbano

Berzovini

et al. 2020

IJERPH

Self Cite

181

122

View full text Add to dashboard Cite

show abstract

“…Such a problem can not be tackled directly attempting to hide the mutual information between samples in the same discriminatory class: the non-differentiability of such a measure as well as the introduced computational complexity is a huge obstacle, and NDR proposes itself as a proxy for such a measure. Nonetheless, previous works have already shown that adding further constraints to the learning problem could be effective [7] as, typically, the trained ANN models are over-sized and allows a large number of solutions to the same learning task [8]. Our experiments with NDR show that in practical cases it is possible to strike a good balance between non discriminatory constraint and target performance.…”

Section: Introductionmentioning

confidence: 76%

A non-discriminatory approach to ethical deep learning

Tartaglione¹,

Grangetto²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Artificial neural networks perform state-of-the-art in an ever-growing number of tasks, nowadays they are used to solve an incredibly large variety of tasks. However, typical training strategies do not take into account lawful, ethical and discriminatory potential issues the trained ANN models could incur in.In this work we propose NDR, a non-discriminatory regularization strategy to prevent the ANN model to solve the target task using some discriminatory features like, for example, the ethnicity in an image classification task for human faces. In particular, a part of the ANN model is trained to hide the discriminatory information such that the rest of the network focuses in learning the given learning task. Our experiments show that NDR can be exploited to achieve non-discriminatory models with both minimal computational overhead and performance loss.

show abstract

“…In this work we aim at overcoming the above limitations proposing a regularization method that produces a structured sparsification, focusing on removing entire neurons instead of single parameters. We also leverage our recent research showing that post-synaptic potential regularization is able to boost generalization over other regularizers [31].…”

Section: Related Workmentioning

confidence: 99%

SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networks

Tartaglione¹,

Bragagnolo²,

Odierna³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Deep neural networks include millions of learnable parameters, making their deployment over resource-constrained devices problematic. SeReNe (Sensitivity-based Regularization of Neurons) is a method for learning sparse topologies with a structure, exploiting neural sensitivity as a regularizer. We define the sensitivity of a neuron as the variation of the network output with respect to the variation of the activity of the neuron. The lower the sensitivity of a neuron, the less the network output is perturbed if the neuron output changes. By including the neuron sensitivity in the cost function as a regularization term, we are able to prune neurons with low sensitivity. As entire neurons are pruned rather then single parameters, practical network footprint reduction becomes possible. Our experimental results on multiple network architectures and datasets yield competitive compression ratios with respect to state-of-the-art references.

show abstract

Post-synaptic Potential Regularization Has Potential

Cited by 6 publications

References 20 publications

Unveiling COVID-19 from CHEST X-Ray with Deep Learning: A Hurdles Race with Small Data

Unveiling COVID-19 from CHEST X-Ray with Deep Learning: A Hurdles Race with Small Data

A non-discriminatory approach to ethical deep learning

SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networks

Contact Info

Product

Resources

About