Rethinking Binary Hyperparameters for Deep Transfer Learning

Plested, Jo; Shen, Xuyang; Gedeon, Tom

doi:10.1007/978-3-030-92270-2_40

Cited by 7 publications

(11 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Convolutional Neural Networks (CNNs) represent the predominant deep learning (DL) technique for image analysis [8], [11]. A CNN operates as a feedforward neural network designed to extract image features by applying filters, also referred to as kernels or feature detectors, to the image.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Explainable Deep Learning to Classify Royal Navy Ships

Baesens,

Adams,

Pacheco-Ruiz

et al. 2024

IEEE Access

View full text Add to dashboard Cite

We research how deep learning convolutional neural networks can be used to to automatically classify the unique data set of black-and-white naval ships images from the Wright and Logan photographic collection held by the National Museum of the Royal Navy. We contrast various types of deep learning methods: pretrained models such as ConvNeXt, ResNet and EfficientNet, and ConvMixer. We also thoroughly investigate the impact of data preprocessing and externally obtained images on model performance. Finally, we research how the models estimated can be made transparent using visually appealing interpretability techniques such as Grad-CAM. We find that ConvNeXt has the best performance for our data set achieving an accuracy of 79.62% for 0-notch classification and an impressive 94.86% for 1-notch classification. The results indicate the importance of appropriate image preprocessing. Image segmentation combined with soft augmentation significantly contributes to model performance. We consider this research to be original in several aspects. Notably, it distinguishes itself through the uniqueness of the acquired dataset. Additionally, its distinctiveness extends to the analytical modeling pipeline, which encompasses a comprehensive range of modeling steps, including data preprocessing (incorporating external data, image segmentation, and image augmentation) and the use of deep learning techniques such as ConvNeXt, ResNet, EfficientNet, and ConvMixer. Furthermore, the research employs explanatory tools like Grad-CAM to enhance model interpretability and usability. We believe the proposed methodology offers lots of potential for documenting historic image collections.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

“…Each image was subjected to meticulous manual labeling into various predefined ship types or categories, such as submarines, cruisers, destroyers, carriers, and more. Subsequently, deep learning convolutional neural networks (CNNs) were trained to autonomously recognise these classifications based on image attributes [7], [8].…”

Section: Introductionmentioning

confidence: 99%

Explainable Deep Learning to Classify Royal Navy Ships

Baesens,

Adams,

Pacheco-Ruiz

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In the Deep Learning, Reinforcement Learning and Evolutionary Strategies fields, researchers are most interested in figuring out how to solve a specific set of problems and evaluate the performance of their agents/models with some loss or fitness functions. While their results are often impressive even in the generality of the found solutions (Team et al, 2021) or how prone they are to transfer learning (Liu et al, 2023;Plested and Gedeon, 2022), it is extremely difficult to afterwards take these models and "evolve them" in order to reuse as many parameters as possible and decrease the training time of a new, more complex version of the same model, or for a different task. It is this kind of complexification that we are most interested in with this line of research.…”

Section: Complexification and Open Endednessmentioning

confidence: 99%

Adversarial Reprogramming of Neural Cellular Automata

Randazzo

Mordvintsev

Niklasson

et al. 2021

Distill

View full text Add to dashboard Cite

Neural Cellular Automata (NCA) models have shown remarkable capacity for pattern formation and complex global behaviors stemming from local coordination. However, in the original implementation of NCA, cells are incapable of adjusting their own orientation, and it is the responsibility of the model designer to orient them externally. A recent isotropic variant of NCA (Growing Isotropic Neural Cellular Automata) makes the model orientation-independent -cells can no longer tell up from down, nor left from right -by removing its dependency on perceiving the gradient of spatial states in its neighborhood. In this work, we revisit NCA with a different approach: we make each cell responsible for its own orientation by allowing it to "turn" as determined by an adjustable internal state. The resulting Steerable NCA contains cells of varying orientation embedded in the same pattern. We observe how, while Isotropic NCA are orientationagnostic, Steerable NCA have chirality: they have a predetermined left-right symmetry. We therefore show that we can train Steerable NCA in similar but simpler ways than their Isotropic variant by: (1) breaking symmetries using only two seeds, or (2) introducing a rotation-invariant training objective and relying on asynchronous cell updates to break the up-down symmetry of the system.

show abstract

“…Fully supervised computer vision models for problems like classification are typically trained on datasets like ImageNet (Deng et al 2009a), OpenImages (Kuznetsova et al 2018), JFT300M (Kolesnikov et al 2020;Sun et al 2017) etc., and have also proven themselves to be effective for a variety of downstream tasks via transfer learning (Plested and Gedeon 2022;Guo et al 2019;Wan et al 2019). Despite this, it is challenging to adapt these models to other domains due to various reasons including limited data and annotation overhead.…”

Section: Introductionmentioning

confidence: 99%

CoPL: Contextual Prompt Learning for Vision-Language Understanding

Goswami,

Karanam,

Udhayanan

et al. 2024

AAAI

View full text Add to dashboard Cite

Recent advances in multimodal learning has resulted in powerful vision-language models, whose representations are generalizable across a variety of downstream tasks. Recently, their generalization ability has been further extended by incorporating trainable prompts, borrowed from the natural language processing literature. While such prompt learning techniques have shown impressive results, we identify that these prompts are trained based on global image features which limits itself in two aspects: First, by using global features, these prompts could be focusing less on the discriminative foreground image, resulting in poor generalization to various out-of-distribution test cases. Second, existing work weights all prompts equally whereas intuitively, prompts should be reweighed according to the semantics of the image. We address these as part of our proposed Contextual Prompt Learning (CoPL) framework, capable of aligning the prompts to the localized features of the image. Our key innovations over earlier works include using local image features as part of the prompt learning process, and more crucially, learning to weight these prompts based on local features that are appropriate for the task at hand. This gives us dynamic prompts that are both aligned to local image features as well as aware of local contextual relationships. Our extensive set of experiments on a variety of standard and few-shot datasets show that our method produces substantially improved performance when compared to the current state of the art methods. We also demonstrate both few-shot and out-of-distribution performance to establish the utility of learning dynamic prompts that are aligned to local image features.

show abstract

Rethinking Binary Hyperparameters for Deep Transfer Learning

Cited by 7 publications

References 22 publications

Explainable Deep Learning to Classify Royal Navy Ships

Explainable Deep Learning to Classify Royal Navy Ships

Adversarial Reprogramming of Neural Cellular Automata

CoPL: Contextual Prompt Learning for Vision-Language Understanding

Contact Info

Product

Resources

About