Neural Architecture Search (NAS) is a promising and rapidly evolving research area. Training a large number of neural networks requires an exceptional amount of computational power, which makes NAS unreachable for those researchers who have limited or no access to high-performance clusters and supercomputers. A few benchmarks with precomputed neural architectures performances have been recently introduced to overcome this problem and ensure more reproducible experiments. However, these benchmarks are only for the computer vision domain and, thus, are built from the image datasets and convolution-derived architectures. In this work, we step outside the computer vision domain by leveraging the language modeling task, which is the core of natural language processing (NLP). Our main contribution is as follows: we have provided search space of recurrent neural networks on the text datasets and trained 14k architectures within it; we have conducted both intrinsic and extrinsic evaluation of the trained models using datasets for semantic relatedness and language understanding evaluation; finally, we have tested several NAS algorithms to demonstrate how the precomputed results can be utilized. We believe that our results have high potential of usage for both NAS and NLP communities.

Developmental noise—which level may vary within a certain backlash allowed by natural selection—is a reflection of the state of a developing system or developmental stability. Phenotypic variations inside the genetically determined norm observed in case of fluctuating asymmetry provide a unique opportunity for evaluating this form of ontogenetic variability. Low levels of developmental noise for the biologic system under study is observed under certain conditions, while its increase acts as a measure of stress. The concordance of changes in developmental stability with changes in other parameters of developmental homeostasis indicates the significance of fluctuating asymmetry estimates. All this determines the future prospects of the study of fluctuating asymmetry not only for developmental biology, but also for population biology. The study of developmental stability may act as the basis of an approach of population developmental biology to assess the nature of the phenotypic diversity and the state of natural populations under various impacts and during evolutionary transformations.

A population-phenogenetic analysis of plants (European white birch) based on investigation of the stability of their development in natural populations was performed. The data obtained allow character izing the violation of the stability in natural populations as a response of developing organisms to environ mental (climate) changes not only in the reclamation of new high altitude areas, but during temperature increases in plains as well.Keywords: stability of development, European white birch, climate change, population-phenogenetic anal ysis, ecological periphery of an area.

Neural architecture search (NAS) targets at finding the optimal architecture of a neural network for a problem or a family of problems. Evaluations of neural architectures are very time-consuming. One of the possible ways to mitigate this issue is to use low-fidelity evaluations, namely training on a part of a dataset, fewer epochs, with fewer channels, etc. In this paper, we propose to improve low-fidelity evaluations of neural architectures by using a knowledge distillation. Knowledge distillation adds to a loss function a term forcing a network to mimic some teacher network. We carry out experiments on CIFAR-100 and ImageNet and study various knowledge distillation methods. We show that training on the small part of a dataset with such a modified loss function leads to a better selection of neural architectures than training with a logistic loss. The proposed low-fidelity evaluations were incorporated into a multi-fidelity search algorithm that outperformed the search based on high-fidelity evaluations only (training on a full dataset).

No abstract