AutoFCL: automatically tuning fully connected layers for handling small dataset

Basha, S. H. Shabbeer; Vinakota, Sravan Kumar; Dubey, Shiv Ram; Pulabaigari, Viswanath; Mukherjee, Snehasis

doi:10.1007/s00521-020-05549-4

Cited by 26 publications

(13 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, three main approaches for deep learning systems training are: manual architecture construction, transfer-learning, and neural architecture search [ 39 , 40 , 41 ]. We used transfer learning alongside manually created networks to assess the importance of wavebands triplets in previous research.…”

Section: Methodsmentioning

confidence: 99%

Multi-Class CNN for Classification of Multispectral and Autofluorescence Skin Lesion Clinical Images

Lihacova

Bondarenko

Chizhov

et al. 2022

JCM

View full text Add to dashboard Cite

In this work, we propose to use an artificial neural network to classify limited data of clinical multispectral and autofluorescence images of skin lesions. Although the amount of data is limited, the deep convolutional neural network classification of skin lesions using a multi-modal image set is studied and proposed for the first time. The unique dataset consists of spectral reflectance images acquired under 526 nm, 663 nm, 964 nm, and autofluorescence images under 405 nm LED excitation. The augmentation algorithm was applied for multi-modal clinical images of different skin lesion groups to expand the training datasets. It was concluded from saliency maps that the classification performed by the convolutional neural network is based on the distribution of the major skin chromophores and endogenous fluorophores. The resulting classification confusion matrices, as well as the performance of trained neural networks, have been investigated and discussed.

show abstract

Section: Methodsmentioning

confidence: 99%

Multi-Class CNN for Classification of Multispectral and Autofluorescence Skin Lesion Clinical Images

Lihacova

Bondarenko

Chizhov

et al. 2022

JCM

View full text Add to dashboard Cite

show abstract

“…In this section, we briefly review the most relevant researches on: (1) color detection [ 8 – 14 ]; and (2) flower classification using deep learning [ 3 , 5 , 15 – 20 ].…”

Section: Methodsmentioning

confidence: 99%

“…A comparison of the performance of some deep learning architectures is also made in the work by Basa et al [ 20 ]. In this research, they compared the performance of VGG16, ResNet-50, MobileNet, DenseNet, and NasNet-Mobile combined with a fine-tuning method deploying some datasets including the Oxford-102 Flower dataset.…”

Section: Methodsmentioning

confidence: 99%

Automated color detection in orchids using color labels and deep learning

et al. 2021

View full text Add to dashboard Cite

The color of particular parts of a flower is often employed as one of the features to differentiate between flower types. Thus, color is also used in flower-image classification. Color labels, such as ‘green’, ‘red’, and ‘yellow’, are used by taxonomists and lay people alike to describe the color of plants. Flower image datasets usually only consist of images and do not contain flower descriptions. In this research, we have built a flower-image dataset, especially regarding orchid species, which consists of human-friendly textual descriptions of features of specific flowers, on the one hand, and digital photographs indicating how a flower looks like, on the other hand. Using this dataset, a new automated color detection model was developed. It is the first research of its kind using color labels and deep learning for color detection in flower recognition. As deep learning often excels in pattern recognition in digital images, we applied transfer learning with various amounts of unfreezing of layers with five different neural network architectures (VGG16, Inception, Resnet50, Xception, Nasnet) to determine which architecture and which scheme of transfer learning performs best. In addition, various color scheme scenarios were tested, including the use of primary and secondary color together, and, in addition, the effectiveness of dealing with multi-class classification using multi-class, combined binary, and, finally, ensemble classifiers were studied. The best overall performance was achieved by the ensemble classifier. The results show that the proposed method can detect the color of flower and labellum very well without having to perform image segmentation. The result of this study can act as a foundation for the development of an image-based plant recognition system that is able to offer an explanation of a provided classification.

show abstract

“…Victoria et al [29] used the BO algorithm to optimize hyperparameters to enhance the performance of a CNN model. In [15] and [16], Basha et al presented a method for autotuning hyperparameters using Bayesian optimization. The scope of [15] is limited to autotune the FC layers of CNN models while [16] automatically tuned some CNN layers as well.…”

Section: Related Workmentioning

confidence: 99%

“…NAS methods find the structure of the model from scratch and hence require a lot of GPU hours. Basha et al [15] and [16] proposed a method to autotune CNN models for improved transfer learning using a Bayesian optimization technique. Motivated by the success of [15] for tuning FC layers following a CNN, in terms of accuracy and GPU hours, we apply the TPE algorithm, which is a variant of Bayesian optimization, to fine-tune FC layers following a self-supervised model.…”

Section: Introductionmentioning

confidence: 99%

Autotuned Classification Based on Knowledge Transferred from Self-supervised Models

Kishore

Mukherjee

2022

Preprint

View full text Add to dashboard Cite

With the recent advancements of deep learning-based methods in image classification, the requirement of a huge amount of training data is inevitable to avoid overfitting problems. Moreover, supervised deep learning models require labeled datasets. Preparing such a huge amount of labeled data takes a lot of human effort and time. In this scenario, self-supervised models are becoming popular because of their ability to learn even from unlabelled datasets. This paper presents a method to automatically tune the hyperparameters of a self-supervised model for image classification. The learned features are transferred from a self-supervised model to a set of Fully Connected (FC) layer, where the hyperparameters related to the architecture are tuned. A softmax layer is used following the FC layers for classification. The hyperparameters such as the number of layers, number of units in each layer, learning rate, dropout, and optimizer are automatically tuned in these FC layers, using a Bayesian optimization technique called the Tree-structured Parzen Estimator Approach (TPE) algorithm. To evaluate the performance of the proposed method, state-of-the-art self-supervised models such as Simclr and SWAV are used to extract the learned features. Experiments are carried out on CIFAR-10, CIFAR-100, and Tiny Imagenet datasets. The proposed method outperforms the state-of-the-art.

show abstract

AutoFCL: automatically tuning fully connected layers for handling small dataset

Cited by 26 publications

References 41 publications

Multi-Class CNN for Classification of Multispectral and Autofluorescence Skin Lesion Clinical Images

Multi-Class CNN for Classification of Multispectral and Autofluorescence Skin Lesion Clinical Images

Automated color detection in orchids using color labels and deep learning

Autotuned Classification Based on Knowledge Transferred from Self-supervised Models

Contact Info

Product

Resources

About