Directing at various problems of the traditional Q-Learning algorithm, such as heavy repetition and disequilibrium of explorations, the reinforcement-exploration strategy was used to replace the decayed ε-greedy strategy in the traditional Q-Learning algorithm, and thus a novel self-adaptive reinforcement-exploration Q-Learning (SARE-Q) algorithm was proposed. First, the concept of behavior utility trace was introduced in the proposed algorithm, and the probability for each action to be chosen was adjusted according to the behavior utility trace, so as to improve the efficiency of exploration. Second, the attenuation process of exploration factor ε was designed into two phases, where the first phase centered on the exploration and the second one transited the focus from the exploration into utilization, and the exploration rate was dynamically adjusted according to the success rate. Finally, by establishing a list of state access times, the exploration factor of the current state is adaptively adjusted according to the number of times the state is accessed. The symmetric grid map environment was established via OpenAI Gym platform to carry out the symmetrical simulation experiments on the Q-Learning algorithm, self-adaptive Q-Learning (SA-Q) algorithm and SARE-Q algorithm. The experimental results show that the proposed algorithm has obvious advantages over the first two algorithms in the average number of turning times, average inside success rate, and number of times with the shortest planned route.

In MOOC learning, learners’ emotions have an important impact on the learning effect. In order to solve the problem that learners’ emotions are not obvious in the learning process, we propose a method to identify learner emotion by combining eye movement features and scene features. This method uses an adaptive window to partition samples and enhances sample features through fine-grained feature extraction. Using an adaptive window to partition samples can make the eye movement information in the sample more abundant, and fine-grained feature extraction from an adaptive window can increase discrimination between samples. After adopting the method proposed in this paper, the four-category emotion recognition accuracy of the single modality of eye movement reached 65.1% in MOOC learning scenarios. Both the adaptive window partition method and the fine-grained feature extraction method based on eye movement signals proposed in this paper can be applied to other modalities.

No abstract

Long non-coding RNAs (lncRNAs) exert impacts on multiple fundamental and important biological processes. Many lncRNAs have been functionally associated with cancers. Utilizing experimental and bioinformatics approaches to identify and annotate lncRNAs with cancer-associated roles is laborious and time-consuming. Therefore, more and more researchers have focused on computational methods as an alternative candidate to find out unknown associations between lncRNAs and diseases, in particular, cancers. In this study, under the situation that there were few known lncRNA-disease associations out of huge unknown associations, we explored a novel two-stage prediction model (namely DRW-BNSP) for inferring lncRNA-disease associations: In the first stage, we designed a Dual Random Walk (DRW) model to obtain the primary prediction scores by walking on two combined similarity networks which were reconstructed; In the second stage, we used a Bipartite Network Space Projection (BNSP) model to make the primary prediction scores to be more accurate furtherly. Compared with other state-of-the-art methods in similar type, our DRW-BNSP could not only function on new lncRNAs and isolated diseases, but also achieve higher AUC value of 0.9344 and 0.9432 on the first dataset (namely Dataset1) and second dataset (namely Dataset2) built by us. Furthermore, case study further confirmed the predictive dependability of our DRW-BNSP for inferring potential lncRNA-disease associations.