Compression techniques for deep neural network models are becoming very important for the efficient execution of high-performance deep learning systems on edge-computing devices. The concept of model compression is also important for analyzing the generalization error of deep learning, known as the compression-based error bound. However, there is still huge gap between a practically effective compression method and its rigorous background of statistical learning theory. To resolve this issue, we develop a new theoretical framework for model compression and propose a new pruning method called {\it spectral pruning} based on this framework. We define the ``degrees of freedom'' to quantify the intrinsic dimensionality of a model by using the eigenvalue distribution of the covariance matrix across the internal nodes and show that the compression ability is essentially controlled by this quantity. Moreover, we present a sharp generalization error bound of the compressed model and characterize the bias--variance tradeoff induced by the compression procedure. We apply our method to several datasets to justify our theoretical analyses and show the superiority of the the proposed method.

No abstract

Caterpillars (Lepidoptera and Symphyta larvae) employ diverse visual defensive tactics, and effectiveness of such tactics may be highly dynamic across time due to seasonal changes in the predator assemblages and their preferences. However, this has rarely been studied especially in tropical regions. Here we assessed temporal changes in the defensive value of caterpillar color and shape, using six types of plasticine dummy caterpillars: three colors (green, black, and white) × two shapes (curled and straight). These dummy caterpillars were deployed five times over different seasons in tropical forests of Xishuangbanna (China) and, as a comparison, twice in a temperate forest of Hirosaki (Japan). The colors and shapes of dummy caterpillars simulate visual traits of black sawfly larvae which take the curled resting posture in tropical rainforests of Xishuangbanna, apparently masquerading excrements commonly found on plants, while in Hirosaki there is no black‐curled sawfly larvae and few excrements on plants. We found no significant effects of caterpillar colors or shapes on predation in Hirosaki. In contrast, black and curled caterpillars received significantly lower predation by birds in Xishuangbanna constantly across time. However, we were unable to provide evidence that the black‐curled sawfly larvae are masquerading as excrements. Shapes of the dummy caterpillars also affected the predation by ants and parasitoid wasps at certain times. This is the first report on ecological function of the curled posture of sawfly larvae, and we demonstrated the importance to assess the temporal dynamics of predation and effectiveness of defensive tactics in tropical forests.

The model size of deep neural network is getting larger and larger to realize superior performance in complicated tasks. This makes it difficult to implement deep neural network in small edge-computing devices. To overcome this problem, model compression methods have been gathering much attention. However, there have been only few theoretical back-grounds that explain what kind of quantity determines the compression ability. To resolve this issue, we develop a new theoretical frame-work for model compression, and propose a new method called Spectral-Pruning based on the theory. Our theoretical analysis is based on the observation such that the eigenvalues of the covariance matrix of the output from nodes in the internal layers often shows rapid decay. We define "degree of freedom" to quantify an intrinsic dimensionality of the model by using the eigenvalue distribution and show that the compression ability is essentially controlled by this quantity. Along with this, we give a generalization error bound of the compressed model. Our proposed method is applicable to wide range of models, unlike the existing methods, e.g., ones possess complicated branches as implemented in SegNet and ResNet. Our method makes use of both "input" and "output" in each layer and is easy to implement. We apply our method to several datasets to justify our theoretical analyses and show that the proposed method achieves the state-of-the-art performance.

Stent-like complicated cylindrical structures were made with pipes of stainless-steel SUS 304 using new lithography and wet chemical etching. In the new lithography, patterns on a flat reticle were printed on a pipe coated with a resist film by synchronously scanning the reticle linearly and rotating the pipe around the axis, and limiting the momentary exposure area on the top ridge of the pipe by placing an oblong slit on the reticle in parallel to the pipe axis. The patterned pipe was wetly etched in an aqueous solution of ferric chloride using the resist patterns as etching masks. Because the etching was progressed equally in all the directions from the resist pattern edges, masked parts under the resist patterns were also gradually etched from the pattern edges and undercut during the pipe was penetrated through the wall. Caused by the undercut, obtained mesh widths became narrower than the resist pattern widths. However, a stent-like mesh structure with widths of 108±12.4 (3σ) µm was decently fabricated by appropriately controlling the resist pattern width and the etching time. To attain higher accuracy in the future, cross section profiles of the mesh and relationship between resist pattern widths and mesh widths were discussed in detail.