Towards Complex Features: Competitive Receptive Fields in Unsupervised Deep Networks

Hankins, Richard; Peng, Yao; Yin, Hujun

doi:10.1007/978-3-030-03493-1_87

Cited by 4 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work we introduced the modular building blocks of CSNNs to learn representations in an unsupervised manner [15] 83.35 90.59 [29] 78.57 (SVM) -----------K-means (Triangle) (best single model accuracy we know) [30] 79.60 (SVM) Step 0…”

Section: Discussionmentioning

confidence: 99%

“…Chan et al [28] used Principal Component Analysis (PCA) in a convolutional manner to learn two-stage filter banks, which are followed by binary hashing and block histograms to compress the representation. In a similar way Hankins et al [29] used SOMs and additionally introduced pooling layers to shrink SOMactivations along the feature dimension. Coates et al [30] studied hyperparameter choices and preprocessing strategies on several single-layer networks learned on convolutional patches, including sparse auto-encoders, sparse restricted Boltzmann machines, K-means clustering, and Gaussian mixture models and showed, e.g., general performance gains when the representation size is increased.…”

Section: B Unsupervised Backpropagation-free Representation Learningmentioning

confidence: 99%

See 1 more Smart Citation

CSNNs: Unsupervised, Backpropagation-Free Convolutional Neural Networks for Representation Learning

Stuhr

Bräuer²

2019

2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA)

View full text Add to dashboard Cite

This work combines Convolutional Neural Networks (CNNs), clustering via Self-Organizing Maps (SOMs) and Hebbian Learning to propose the building blocks of Convolutional Self-Organizing Neural Networks (CSNNs), which learn representations in an unsupervised and Backpropagation-free manner. Our approach replaces the learning of traditional convolutional layers from CNNs with the competitive learning procedure of SOMs and simultaneously learns local masks between those layers with separate Hebbian-like learning rules to overcome the problem of disentangling factors of variation when filters are learned through clustering. We investigate the learned representation by designing two simple models with our building blocks, achieving comparable performance to many methods which use Backpropagation, while we reach comparable performance on Cifar10 and give baseline performances on Cifar100, Tiny ImageNet and a small subset of ImageNet for Backpropagationfree methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: B Unsupervised Backpropagation-free Representation Learningmentioning

confidence: 99%

CSNNs: Unsupervised, Backpropagation-Free Convolutional Neural Networks for Representation Learning

Stuhr

Bräuer²

2019

2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA)

View full text Add to dashboard Cite

show abstract

“…The continuous advancement of deep network architectures in image classification generates networks adaptable for face recognition. The study of Dong et al [156], Bruna and Mallat [157], and Hankins et al [158] are some image classification networks with prospect for face classification. Hence, face recognition will remain an active research striving for sophisticated frameworks.…”

Section: Basic Challenges For Face Recognitionmentioning

confidence: 99%

Deep-learned faces: a survey

Arachchilage

Izquierdo

2020

J Image Video Proc.

View full text Add to dashboard Cite

Deep learning technology has enabled successful modeling of complex facial features when high-quality images are available. Nonetheless, accurate modeling and recognition of human faces in real-world scenarios “on the wild” or under adverse conditions remains an open problem. Consequently, a plethora of novel deep network architectures addressing issues related to low-quality images, varying pose, illumination changes, emotional expressions, etc., have been proposed and studied over the last few years.This survey presents a comprehensive analysis of the latest developments in the field. A conventional deep face recognition system entails several main components: deep network, optimization loss function, classification algorithm, and train data collection. Aiming at providing a complete and comprehensive study of such complex frameworks, this paper first discusses the evolution of related network architectures. Next, a comparative analysis of loss functions, classification algorithms, and face datasets is given. Then, a comparative study of state-of-the-art face recognition systems is presented. Here, the performance of the systems is discussed using three benchmarking datasets with increasing degrees of complexity. Furthermore, an experimental study was conducted to compare several openly accessible face recognition frameworks in terms of recognition accuracy and speed.

show abstract

“…As a result, simpler models are achieved, and hence, the interpretability is better. Deep learning is nowadays on the rise (Hankins, Peng, & Yin, 2018) and could also be benefited from the data selection step.…”

Section: Applying a Data Preprocessing Phasementioning

confidence: 99%

Metaheuristic algorithm to train product and sigmoid neural network classifiers

Tallón‐Ballesteros

2019

Expert Systems

View full text Add to dashboard Cite

This paper develops three frameworks based on a metaheuristic algorithm to train neural network classifiers. The architecture is a single‐hidden‐layer feedforward network. The first methodology spreads a base configuration over the nodes of a computing cluster; each of them executes the same algorithm to train the neural network with a different parameter setting. The second approach does a refined training via a biphase metaheuristic algorithm to maintain the diversity a period longer than the usual; it may be run in a sequential or distributed way. The third framework performs a data preparation phase by means of feature subset selection to reduce the number of inputs to the biphase metaheuristic algorithm. The two first methodologies have been tested using a complete test bed with product and unipolar sigmoid units in the hidden layer, and the statistical tests reveal that product nodes are significantly the most accurate. The third framework has included four feature subset selectors with different properties to reduce the number of inputs to the product unit artificial neural network, and the nonstatistical test shed light on that the results with a preprocessing phase are significantly more accurate than the results with the raw data.

show abstract

Towards Complex Features: Competitive Receptive Fields in Unsupervised Deep Networks

Cited by 4 publications

References 14 publications

CSNNs: Unsupervised, Backpropagation-Free Convolutional Neural Networks for Representation Learning

CSNNs: Unsupervised, Backpropagation-Free Convolutional Neural Networks for Representation Learning

Deep-learned faces: a survey

Metaheuristic algorithm to train product and sigmoid neural network classifiers

Contact Info

Product

Resources

About