Self-organizing state aggregation for architecture design of Q-learning

Hwang, Kao-Shing; Lin, Hsin-Yi; Hsu, Yuan-Pao; Yu, Hung-Hsiu

doi:10.1016/j.ins.2011.02.017

Cited by 27 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The above process makes the learning strategy of RL have long-term effects. Q-learning [6,19] is one of the famous RL approaches. Its goal is for an agent to learn the optimal long-term expected reward value Q(s, a) for each pair of state (s) and action (a).…”

Section: Background and Related Work 21 Memetic Algorithms And Q-leamentioning

confidence: 99%

A Q-learning-based memetic algorithm for multi-objective dynamic software project scheduling

Shen

Minku

Marturi

et al. 2018

Information Sciences

View full text Add to dashboard Cite

Software project scheduling is the problem of allocating employees to tasks in a software project. Due to the large scale of current software projects, many studies have investigated the use of optimization algorithms to find good software project schedules. However, despite the importance of human factors to the success of software projects, existing work has considered only a limited number of human properties when formulating software project scheduling as an optimization problem. Moreover, the changing environments of software companies mean that software project scheduling is a dynamic optimization problem. However, there is a lack of effective dynamic scheduling approaches to solve this problem. This work proposes a more realistic mathematical model for the dynamic software project scheduling problem. This model considers that skill proficiency can improve over time and, different from previous work, it considers that such improvement is affected by the employees' properties of motivation and learning ability, and by the skill difficulty. It also defines the objective of employees' satisfaction with the allocation. It is considered together with the objectives of project duration, cost, robustness and stability under a variety of practical constraints. To adapt schedules to the dynamically changing software project environments, a multi-objective two-archive memetic algorithm based on Q-learning (MOTAMAQ) is proposed to solve the problem in a proactive-rescheduling way. Different from previous work, MOTAMAQ learns the most appropriate global and local search methods to be used for different software project environment states by using Q-learning. Extensive experiments on 18 dynamic benchmark instances and 3 instances derived from real-world software projects were performed. A comparison with seven other meta-heuristic algorithms shows that the strategies used by our novel approach are very effective in improving its convergence performance in dynamic environments, while maintaining a good distribution and spread of solutions. The Q-learning-based learning mechanism can choose appropriate search operators for the different scheduling environments. We also show how different trade-offs among the five objectives can provide software managers with a deeper insight into various compromises among many objectives, and enabling them to make informed decisions.

show abstract

Section: Background and Related Work 21 Memetic Algorithms And Q-leamentioning

confidence: 99%

A Q-learning-based memetic algorithm for multi-objective dynamic software project scheduling

Shen

Minku

Marturi

et al. 2018

Information Sciences

View full text Add to dashboard Cite

show abstract

“…As a result, developing some basic behaviors before constructing the proposed system is required. Imitation is one method for humans learning new behaviors and this idea inspired the reinforcement learning based (RL-based) decision tree (RLbased DT) proposed in our previous work [22]. First, the instructor controls the robot based on his/her instincts or experience, with the robot recording the patterns from its sensory inputs, denoted as attributes, and the robot's outputs, denoted as classes.…”

Section: A Individual Behavior Recorded By a Reinforcement Learning mentioning

confidence: 99%

A Simple Scheme for Formation Control Based on Weighted Behavior Learning

Lin

Hwang

Wang

2014

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Several correlated issues of autonomy and simplicity regarding formation control for robots with a self-awareness mechanism in unstructured environments are considered. To achieve autonomy and simplicity, a hybrid scheme is derived for robot maneuvering based on a multibehavioral system. The system holds some self-awareness capabilities ensuring precision and robustness in the presence of internal and external disturbances within the limited capacity of interrobot communication. This is to ensure that the robots can march and simultaneously maintain their assigned formation and avoid hazardous collisions on the way to their destination. These self-awareness capabilities are achieved through a layered reinforcement learning algorithm. At the bottom level, robots are equipped with a set of primitive behaviors learned prior to team formation. The high-level combined behavior is generated by multiplying the outputs of each primitive behavior by its weight, and then summing and normalizing the results. The weights keep adaptively adjusting to the proposed reinforcement learning method. Once a robot receives a command providing the formation shape and the location of the destination, the robot approaches the destination autonomously and keeps an appropriate distance from its neighbors to maintain the assigned pattern. Leadership was given to a robot occupying the lead position. The volunteer leader takes the responsibility for keeping the formation, reporting its existence to the other robots, and resetting the position assignment after passing obstacles. Simulations show the practicality and performance of the proposed approach in both static and dynamic obstacle environments.

show abstract

“…Recently, RL methods have gained wide popularity in adaptive control problems with successful applications [14][15][16][17][18][19]. In engineering applications, RL is known as a bio-inspired machine learning technique used to solve sequential decision problems [20][21][22].…”

Section: Introductionmentioning

confidence: 99%