No abstract

Weight tying is now a common setting in many language generation tasks such as language modeling and machine translation. However, a recent study reveals that there is a potential flaw in weight tying. They find that the learned word embeddings are likely to degenerate and lie in a narrow cone when training a language model. They call it the representation degeneration problem and propose a cosine regularization to solve it. Nevertheless, we prove that the cosine regularization is insufficient to solve the problem, as the degeneration is still likely to happen under certain conditions. In this paper, we revisit the representation degeneration problem and theoretically analyze the limitations of the previously proposed solution. Afterward, we propose an alternative regularization method called Laplacian regularization to tackle the problem. Experiments on language modeling demonstrate the effectiveness of the proposed Laplacian regularization.

The physical and mechanical properties are key indexes for determining the quality of particleboards. For this reason, a study on evaluating the physical and mechanical properties of particleboard via a new method has considerable value. Thus, a method based on principal component regression (PCR) analysis and random forest (RF) is proposed in this paper. First, the problems requiring resolution are described after analyzing the production process parameters as well as the physical and mechanical properties of particleboard. Then, an analysis and prediction models based on the PCR and RF method is established. On the basis of the PCR method, the key process parameters that affect various physical and mechanical properties are determined. Based on the RF method, the analysis and prediction model are built from the previously determined process parameters of the physical and mechanical properties. Finally, through experimental analysis, the effectiveness of the analysis and prediction models based on the PCR and RF method are verified. This work was able to determine the relationship between the process parameters and the physical and mechanical properties, which can help improve practical industrial manufacturing effectivity.

As an extension of Dempster–Shafer (D-S) theory, the evidential reasoning (ER) rule can be used as a combination strategy in ensemble learning to deeply mine classifier information through decision-making reasoning. The weight of evidence is an important parameter in the ER rule, which has a significant effect on the result of ensemble learning. However, current research results on the weight of evidence are not ideal, leveraging expert knowledge to assign weights leads to the excessive subjectivity, and using sample statistical methods to assign weights relies too heavily on the samples, so the determined weights sometimes differ greatly from the actual importance of the attributes. Therefore, to solve the problem of excessive subjectivity and objectivity of the weights of evidence, and further improve the accuracy of ensemble learning based on the ER rule, we propose a novel combination weighting method to determine the weight of evidence. The combined weights are calculated by leveraging our proposed method to combine subjective and objective weights of evidence. The regularization of these weights is studied. Then, the evidential reasoning rule is used to integrate different classifiers. Five case studies of image classification datasets have been conducted to demonstrate the effectiveness of the combination weighting method.

No abstract