Hierarchical Reinforcement Learning (HRL) enables autonomous decomposition of challenging long-horizon decision-making tasks into simpler subtasks. During the past years, the landscape of HRL research has grown profoundly, resulting in copious approaches. A comprehensive overview of this vast landscape is necessary to study HRL in an organized manner. We provide a survey of the diverse HRL approaches concerning the challenges of learning hierarchical policies, subtask discovery, transfer learning, and multi-agent learning using HRL. The survey is presented according to a novel taxonomy of the approaches. Based on the survey, a set of important open problems is proposed to motivate the future research in HRL. Furthermore, we outline a few suitable task domains for evaluating the HRL approaches and a few interesting examples of the practical applications of HRL in the Supplementary Material.

Ageing in place demands a new paradigm of inhouse caregiving allowing many aspects of daily lives to be tackled by smart appliances and technologies. The important challenges include the effective provision of recommendations by multiple parties of caregiver constituting changes of the user's behavior. In this multiagent environment, interdependencies between agents become major issues to tackle. This paper presents an approach of dynamic group formation for autonomous caregiving agents to collaborate in recommending different aspects of well-being. The approach supports the agents to regulate the timing of their recommendations, prevent conflicting messages, and cooperate to make more effective persuasions. A simulation of virtual elderly-care system demonstrates how dynamically grouped and collaborating agents can imply improvements in persuasive recommendation.

No abstract

No abstract

a b s t r a c tEpisodic memory is a significant part of cognition for reasoning and decision making. Retrieval in episodic memory depends on the order relationships of memory items which provides flexibility in reasoning and inferences regarding sequential relations for spatio-temporal domain. However, it is still unclear how they are encoded and how they differ from representations in other types of memory like semantic or procedural memory. This paper presents a neural model of sequential representation and inferences on episodic memory. It contrasts with the common views on sequential representation in neural networks that instead of maintaining transitions between events to represent sequences, they are represented as patterns of activation profiles wherein similarity matching operations support inferences and reasoning. Using an extension of multi-channel multi-layered adaptive resonance theory (ART) network, it is shown how episodic memory can be formed and learnt so that the memory performance becomes dependent on the order and the interchange of memory cues. We present experiments as a proof of concepts to show that the model contrasts sequential representations in semantic memory with those in episodic memory and the model can exhibit transitive inferences consistent with human and animals data.