Contributions to Techniques for Learning Non-Reactive Behaviour from Observation

Abstract

Learning from observation allows an expert to train a software agent or robot without explicitly programming the behaviour. Behaviour learned can be broken down into categories: reactive and non-reactive. Actions in reactive behaviour are based on the current environment state whereas actions in non-reactive uses both current state, and any past action or states. We will analyze and compare using a common benchmark two recent and partially studied approaches to learning non-reactive behaviour from observation: Dynamic Bayesian Networks (DBN) and Temporal Backtracking (TB). The goal is to characterize situations where one approach should be preferred over the other. We will attempt to provide a more general case-based reasoning framework for non-reactive behaviour learning. Using the framework and the benchmark, we will analyze and compare three dierent metrics for comparing cases: run similarity, edit distance and Jaccard distance. This will allow characterization of situations where one metric should outperform the other.

A Comparison of Case-Based Reasoning and Probabilistic Graphical Models in the Context of Learning from Observation

“…Once the framework was unified, one of the next steps that was accomplished was to identify the reason for the poor performance of the DBN in learning statebased behavior in the vacuum cleaner domain in [8]. We realized that the DBN learner would store the memory of the action it last performed, and use it to predict the action at the next time step.…”

“…To ensure that the agent and expert are always on the same trajectory, the agent's action is intercepted and replaced by the expert action. We found that the DBN agent was saving the action that it performed, and this was the cause for the poor reproduction of the DBN learner's results in the vacuum cleaner domain in [8]. Once the fix specified above was implemented, the results that were produced matched those of the original paper and can be seen in [9].…”

“…Since one of the focuses of this thesis is characterizing the performance of CBR and PGM techniques across multiple domains, there was a requirement to test them against each other in domains other than the discrete vacuum cleaner domain. This was also an avenue of future work expressed in [8]. Therefore, models were designed in order to handle mixed domains.…”

“…There was preliminary work done in extending CBR to allow for state-based learning by [8]. Jaccard and Edit distances were used to calculate the similarity between runs.…”

“…The weighted similarity metric is very similar to the one presented in [8], where the author gives decaying weights to the more recent elements in the case.…”