Unsupervised Similarity Based Convolutions for Handwritten Digit Classification

Erkoç, Tuǧba; Eskil, M. Taner

doi:10.1109/siu55565.2022.9864689

Cited by 1 publication

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Convolutional filters are feature detectors that operate on a receptive field. Based on this fact, we extract the filter candidates from the input training images by cropping patches with strides of 1, as in our previous study [47]. However, cutting out patches from training images in this fashion leads to a very high number of candidates that often include no useful information.…”

Section: ) Center Of Gravity Based Candidate Filter Extractionmentioning

confidence: 99%

“…However, cutting out patches from training images in this fashion leads to a very high number of candidates that often include no useful information. The number of the candidates directly affects the filter extraction running time because the similarities are calculated using computationally intensive cross-correlation in [47]. Thus, we propose an elimination process to maintain the set of filter candidates at an acceptable size while preserving crucial information.…”

Section: ) Center Of Gravity Based Candidate Filter Extractionmentioning

confidence: 99%

“…This elimination process is composed of variance and center of gravity (CoG) based elimination procedures. In combination with a reduced candidate set, we also calculate the similarities using dot product instead of cross-correlation, which leads to a huge speed gain (up to 16 times) while improving the accuracy compared to our previous work [47].…”

Section: ) Center Of Gravity Based Candidate Filter Extractionmentioning

confidence: 99%

“…After the first filter is discovered, the algorithm starts judging the remaining candidates one by one against the discovered filters to decide whether they carry features that are different enough to become new filters. To determine whether a candidate C i can become a new filter, similarity scores ⃗ S are calculated for the candidate C i with all previously discovered filters W L of the current layer L. The similarities are calculated with dot product instead of cross-correlation which leads to a large speed gain with the reduced candidate set (up to 16 times) while improving the accuracy compared to [47]. Each discovered filter in layer L wants to represent the remaining candidates.…”

Section: ) Extraction Of Filters From the Candidates Setmentioning

confidence: 99%

“…After all candidates are processed, we apply a normalization routine to the discovered filters as opposed to our previous work [47]. The normalization routine begins by stretching the filter weights to the [-1,1] interval.…”

Section: ) Extraction Of Filters From the Candidates Setmentioning

confidence: 99%

See 4 more Smart Citations

A Novel Similarity Based Unsupervised Technique for Training Convolutional Filters

Erkoç¹,

Eskil²

2023

IEEE Access

View full text Add to dashboard Cite

Achieving satisfactory results with Convolutional Neural Networks (CNNs) depends on how effectively the filters are trained. Conventionally, an appropriate number of filters is carefully selected, the filters are initialized with a proper initialization method and trained with backpropagation over several epochs. This training scheme requires a large labeled dataset, which is costly and time-consuming to obtain. In this study, we propose an unsupervised approach that extracts convolutional filters from a given dataset in a self-organized manner by processing the training set only once without using backpropagation training. The proposed method allows for the extraction of filters from a given dataset in the absence of labels. In contrast to previous studies, we no longer need to select the best number of filters and a suitable filter weight initialization scheme. Applying this method to the MNIST, EMNIST-Digits, Kuzushiji-MNIST, and Fashion-MNIST datasets yields high test performances of 99.19%, 99.39%, 95.03%, and 90.11%, respectively, without applying backpropagation training or using any preprocessed and augmented data.INDEX TERMS Convolutional neural networks, feature extraction, unsupervised learning.

show abstract

Section: ) Center Of Gravity Based Candidate Filter Extractionmentioning

confidence: 99%

Section: ) Center Of Gravity Based Candidate Filter Extractionmentioning

confidence: 99%

Section: ) Center Of Gravity Based Candidate Filter Extractionmentioning

confidence: 99%

Section: ) Extraction Of Filters From the Candidates Setmentioning

confidence: 99%

Section: ) Extraction Of Filters From the Candidates Setmentioning

confidence: 99%

See 3 more Smart Citations