No abstract

Why deep neural networks (DNNs) capable of overfitting often generalize well in practice is a mystery [24]. To find a potential mechanism, we focus on the study of implicit biases underlying the training process of DNNs. In this work, for both real and synthetic datasets, we empirically find that a DNN with common settings first quickly captures the dominant low-frequency components, and then relatively slowly captures the high-frequency ones. We call this phenomenon Frequency Principle (F-Principle). The F-Principle can be observed over DNNs of various structures, activation functions, and training algorithms in our experiments. We also illustrate how the F-Principle help understand the effect of early-stopping as well as the generalization of DNNs. This F-Principle potentially provides insights into a general principle underlying DNN optimization and generalization.

Whole-brain mesoscale mapping of primates has been hindered by large brain size and the relatively low throughput of available microscopy methods. Here, we present an integrative approach that combines primate-optimized tissue sectioning and clearing with ultrahigh-speed, large-scale, volumetric fluorescence microscopy, capable of completing whole-brain imaging of a rhesus monkey at 1 µm × 1 µm × 2.5 µm voxel resolution within 100 hours. A progressive strategy is developed for high-efficiency, long-range tracing of individual axonal fibers through the dataset of hundreds of terabytes, establishing a "Serial sectioning and clearing, 3-dimensional Microscopy, with semi-Automated Reconstruction and Tracing" (SMART) pipeline. This system supports effective connectome-scale mapping of large primates that reveals distinct features of thalamocortical projections of the rhesus monkey brain at the level of individual axonal fibers.