We review the 2014 International Planning Competition (IPC-2014), the eighth in a series of competitions starting in 1998. IPC-2014 was held in three separate parts to assess state-of-the-art in three prominent areas of planning research: the deterministic (classical) part (IPCD), the learning part (IPCL), and the probabilistic part (IPPC). Each part evaluated planning systems in ways that pushed the edge of existing planner performance by introducing new challenges, novel tasks, or both. The competition surpassed again the number of competitors than its predecessor, highlighting the competition’s central role in shaping the landscape of ongoing developments in evaluating planning systems.

Reinforcement learning, while being a highly popular learning technique for agents and multi-agent systems, has so far encountered difficulties when applying it to more complex domains due to scaling-up problems. This paper focuses on the use of domain knowledge to improve the convergence speed and optimality of various RL techniques. Specifically, we propose the use of high-level STRIPS operator knowledge in reward shaping to focus the search for the optimal policy. Empirical results show that the plan-based reward shaping approach outperforms other RL techniques, including alternative manual and MDP-based reward shaping when it is used in its basic form. We show that MDP-based reward shaping may fail and successful experiments with STRIPS-based shaping suggest modifications which can overcome encountered problems. The STRIPS-based method we propose allows expressing the same domain knowledge in a different way and the domain expert can choose whether to define an MDP or STRIPS planning task. We also evaluate the robustness of the proposed STRIPS-based technique to errors in the plan knowledge.

This paper investigates the impact of reward shaping in multi-agent reinforcement learning as a way to incorporate domain knowledge about good strategies. In theory, potentialbased reward shaping does not alter the Nash Equilibria of a stochastic game, only the exploration of the shaped agent. We demonstrate empirically the performance of reward shaping in two problem domains within the context of RoboCup KeepAway by designing three reward shaping schemes, encouraging specific behaviour such as keeping a minimum distance from other players on the same team and taking on specific roles. The results illustrate that reward shaping with multiple, simultaneous learning agents can reduce the time needed to learn a suitable policy and can alter the final group performance.

No abstract

Brain computer interface (BCI) controlled assistive robotic systems have been developed with increasing success with the aim to rehabilitation of patients after brain injury [1] to increase independence and quality of life. While such systems may use surgically implanted invasive sensors, non-invasive alternatives can be better suited due to ease of use, reduced cost, improvements in accuracy and reliability with the advancement of the technology and practicality of use. The consumer grade BCI devices often capable of integrating multiple types of signals, including Electroencephalogram (EEG) and Electromyogram (EMG) signals, as well as basic motion based signals such as gyroscopic data. This paper summarises the development of a BCI controlled robotic system using a non-invasive BCI headset "Muse" [2] and an open source robotic arm, U-Arm [2], to accomplish tasks related to rehabilitation, such as access to resources, adaptability or home use. The resulting system used a combination of EMG and gyroscopic sensor readings to control the arm, which could perform a number of different tasks such as picking/placing objects or assist users in eating. Preliminary work was carried out to capture event driven EEG signals as alternative inputs to the controller. To avoid risks of injury while the device is being used in clinical settings, appropriate measures were incorporated into the software control of the arm. The project was a success in collaboration between the University and the East Kent Hospitals Neuro-Rehabilitation Unit, with the long term goal of testing the system in a clinical environment and up-scaling the system to larger robotic effectors.