Abstract. We present a method for transferring knowledge learned in one task to a related task. Our problem solvers employ reinforcement learning to acquire a model for one task. We then transform that learned model into advice for a new task. A human teacher provides a mapping from the old task to the new task to guide this knowledge transfer. Advice is incorporated into our problem solver using a knowledge-based support vector regression method that we previously developed. This advice-taking approach allows the problem solver to refine or even discard the transferred knowledge based on its subsequent experiences. We empirically demonstrate the effectiveness of our approach with two games from the RoboCup soccer simulator: KeepAway and BreakAway. Our results demonstrate that a problem solver learning to play BreakAway using advice extracted from KeepAway outperforms a problem solver learning without the benefit of such advice.

Abstract. We describe a reinforcement learning system that transfers skills from a previously learned source task to a related target task. The system uses inductive logic programming to analyze experience in the source task, and transfers rules for when to take actions. The target task learner accepts these rules through an advice-taking algorithm, which allows learners to benefit from outside guidance that may be imperfect. Our system accepts a human-provided mapping, which specifies the similarities between the source and target tasks and may also include advice about the differences between them. Using three tasks in the RoboCup simulated soccer domain, we demonstrate that this system can speed up reinforcement learning substantially.

Abstract. We describe an application of inductive logic programming to transfer learning. Transfer learning is the use of knowledge learned in a source task to improve learning in a related target task. The tasks we work with are in reinforcement-learning domains. Our approach transfers relational macros, which are finite-state machines in which the transition conditions and the node actions are represented by first-order logical clauses. We use inductive logic programming to learn a macro that characterizes successful behavior in the source task, and then use the macro for decision-making in the early learning stages of the target task. Through experiments in the RoboCup simulated soccer domain, we show that Relational Macro Transfer via Demonstration (RMT-D) from a source task can provide a substantial head start in the target task.

No abstract

The use of genomic markers in forest tree breeding is expected to improve the response to selection, especially within family. To evaluate the potential improvements from genotyping, we analyzed a large Pinus taeda L. clonal population (1,831 cloned individuals) tested in multiple environments. Of the total, 723 clones from five full-sib families were genotyped using 10,337 single-nucleotide polymorphism markers. Single-step models with genomic and pedigree-based relationships produced similar heritability estimates. Breeding value predictions were greatly improved with inclusion of genomic relationships, even when clonal replication was abundant. The improvement was limited to genotyped individuals and attributable to accounting for the Mendelian sampling effect. Reducing clonal replication by omitting data indicated that genotyping improved breeding values similar to clonal replication. Genomic selection predictive ability (masking phenotypes) was greater for stem straightness (0.68) than for growth traits (0.41 to 0.44). Predictive ability for a new full-sibling family was poorer than when full-sibling relationships were present between model training and validation sets. Species that are difficult to propagate clonally can use genotyping to improve within-family selection. Clonal testing combined with genotyping can produce breeding value accuracies adequate to graft selections directly into deployment orchards without progeny testing. Study Implications Genomic markers can improve the reliability of breeding values, resulting in a more confident ranking of individuals within families. For genotyped individuals, the improvements were comparable to clonal testing. Breeding programs for species that are difficult to propagate clonally should consider genotyping to replace or supplement clonal testing as a means to improve within-family selection. For genomic prediction of breeding values without phenotypes (genomic selection), a robust genetic relationship between model training and validation sets is required. The single-step model allows genotyping a subset of the population and is a straightforward extension of well-established methods.