Smooth Non-increasing Square Spatial Extents of Filters in Convolutional Layers of CNNs for Image Classification Problems

Abstract

The present paper considers an open problem of setting hyperparameters for convolutional neural networks aimed at image classification. Since selecting filter spatial extents for convolutional layers is a topical problem, it is approximately solved by accumulating statistics of the neural network performance. The network architecture is taken on the basis of the MNIST database experience. The eight-layered architecture having four convolutional layers is nearly best suitable for classifying small and medium size images. Image databases are formed of grayscale images whose size range is 28 × 28 to 64 × 64 by step 2. Except for the filter spatial extents, the rest of those eight layer hyperparameters are unalterable, and they are chosen scrupulously based on rules of thumb. A sequence of possible filter spatial extents is generated for each size. Then sets of four filter spatial extents producing the best performance are extracted. The rule of this extraction that allows selecting the best filter spatial extents is formalized with two conditions. Mainly, difference between maximal and minimal extents must be as minimal as possible. No unit filter spatial extent is recommended. The secondary condition is that the filter spatial extents should constitute a non-increasing set. Validation on MNIST and CIFAR- 10 databases justifies such a solution, which can be extended for building convolutional neural network classifiers of colour and larger images.

An Infinitely Scalable Dataset of Single-Polygon Grayscale Images as a Fast Test Platform for Semantic Image Segmentation

“…The represented method of generating an infinitely scalable dataset relies on pseudorandomization of the polygon vertices' number by (1) and coordinates of the vertices by (2) -(4). Inequalities (5) and (6) help in making a polygon of an appropriate form and size, unless the polygon is a triangle. On rare occasions, the triangle can be generated very small or thin reminding an arrow (see Fig.…”

“…So, the question is whether it is possible to test segmentation network architectures much faster in order to find optimal solutions that could be imparted to real-world semantic image segmentation tasks. Such solutions are the number of convolutional layers and their parameters (the size and number of filters) [5,8], max pooling layers [9], parameters of deconvolutional layers (the size and number of filters), and, probably, dropout layers [10]. For example, a plausible purpose is to research on training data of smaller and simpler images so that real-world tasks could inherit close-to-optimal network architectures from them.…”

“…Then, however, the number of their entries is not limited. This is about infinite datasets like EEACL26 [5,9,10] or a possible extended version of MNIST where digits would be drawn by a machine simulating inconstancy of human handwriting. In its turn, an infinite dataset for a toy semantic image segmentation must contain primitive objects whose shape and size will vary dramatically.…”

“…A common semantic segmentation network consists of a downsampling subnetwork, upsampling subnetwork, and a pixel classification layer [1,2,4]. A downsampling subnetwork is stacked of convolutional layers, ReLUs, and max pooling layers [5,6]. The upsampling is executed using the transposed convolutional layer (also commonly referred to as deconvolutional layer) performing the upsampling and filtering at the same time [7].…”

A Dropout Technique Study for the Faster R-CNN Detectors With Pretrained Convolutional Neural Networks for Detecting Very Simple Objects That Can Be Masked

“…Therefore, a pretrained CNN to be included into the Faster R-CNN object detector should not contain a DropOut layer, especially if the CNN is trained much longer with the DropOut layer. An exclusion may be for very simple original image classification datasets (like MNIST [13,14] or EEACL26 [2,4,5]), where overfitting is prevented with two or even more DropOut layers and they do not retard the training process.…”