Abstract. Real-time dynamic programming (RTDP) solves Markov decision processes (MDPs) when the initial state is restricted. By visiting (and updating) only a fraction of the state space, this approach can be used to solve problems with intractably large state space. In order to improve the performance of RTDP, a variant based on symbolic representation was proposed, named sRTDP. Traditional RTDP approaches work best on problems with sparse transition matrices where they can often efficiently achieve ǫ-convergence without visiting all states; however, on problems with dense transition matrices where most states are reachable in one step, the sRTDP approach shows an advantage over traditional RTDP by up to three orders of magnitude, as we demonstrate in this paper. We also specify a new variant of sRTDP based on BRTDP, named sBRTDP, which converges quickly when compared to RTDP variants, since it does less updating by making a better choice of the next state to be visited.

Many real-world decision-theoretic planning problemsare naturally modeled using both continuous state andaction (CSA) spaces, yet little work has provided ex-act solutions for the case of continuous actions. Inthis work, we propose a symbolic dynamic program-ming (SDP) solution to obtain the optimal closed-formvalue function and policy for CSA-MDPs with mul-tivariate continuous state and actions, discrete noise,piecewise linear dynamics, and piecewise linear (or re-stricted piecewise quadratic) reward. Our key contribu-tion over previous SDP work is to show how the contin-uous action maximization step in the dynamic program-ming backup can be evaluated optimally and symboli-cally — a task which amounts to symbolic constrainedoptimization subject to unknown state parameters; wefurther integrate this technique to work with an efficientand compact data structure for SDP — the extendedalgebraic decision diagram (XADD). We demonstrateempirical results on a didactic nonlinear planning exam-ple and two domains from operations research to showthe first automated exact solution to these problems.

No abstract

With the promotion of online education, the adaptive learning system has attracted attention due to its good curriculum recommendation function. The student model is an important interface between the adaptive learning system and the user, reflecting the individual characteristics, knowledge status, and cognitive ability of the student. The accuracy of the information in the student model directly affects the quality of the system recommendation service. The traditional student model only judges students based on the basic information and simple test scores. This paper introduces the self-adaptive item bank and adaptive item selection strategy based on the cognitive diagnosis theory that dynamically detects the students' knowledge and analyzes the state according to the answering habits and knowledge mastering status of different students. This paper analyzes and contrasts a variety of traditional cognitive diagnosis theories and proposes a mixed cognitive diagnosis question bank and a selection strategy model to provide strong support for the construction of student models.

No abstract