The need for high-volume data is one of the challenging requirements of the deep learning methods and it makes it harder to apply deep learning algorithms to domains in which the data sources are limited, in other words small. These domains may vary from medical diagnosis to satellite imaging. The performances of the deep learning methods on small datasets can be improved by the approaches such as data augmentation, ensembling, and transfer learning. In this study, we propose a new approach that utilizes transfer learning and ensemble methods to increase the accuracy rates of convolutional neural networks for classification tasks on small data sets. To this end, we generate different-sized sub-networks by fragmenting an existing large pre-trained network then gather those networks to form an ensemble.For ensemble scoring, we also suggest two new methods. Conducted experiments with the proposed technique, on a randomly sampled Cifar10 small subset dataset, reveals promising results.