Integrated learning for interactive synthetic characters

Abstract

The ability to learn is a potentially compelling and important quality for interactive synthetic characters. To that end, we describe a practical approach to real-time learning for synthetic characters. Our implementation is grounded in the techniques of reinforcement learning and informed by insights from animal training. It simpliÞes the learning task for characters by (a) enabling them to take advantage of predictable regularities in their world, (b) allowing them to make maximal use of any supervisory signals, and (c) making them easy to train by humans.We built an autonomous animated dog that can be trained with a technique used to train real dogs called "clicker training". Capabilities demonstrated include being trained to recognize and use acoustic patterns as cues for actions, as well as to synthesize new actions from novel paths through its motion space.A key contribution of this paper is to demonstrate that by addressing the three problems of state, action, and state-action space discovery at the same time, the solution for each becomes easier. Finally, we articulate heuristics and design principles that make learning practical for synthetic characters.

Teachable robots: Understanding human teaching behavior to build more effective robot learners

“…On the other hand, there have been several reinforcement-based approaches positioned strongly along the exploration dimension. For example, several works allow the human to contribute to the reward function [5,10,13,14,31]. An exploration approach has the benefit that the human need not know exactly how the agent should perform the task, and learning does not require their undivided attention.…”

“…Many question RL as a viable technique for complex real-world environments due to practical problems; but it has certain desirable qualities, like exploring and learning from experience, prompting its use for robots and game characters. A popular approach incorporates real-time human feedback by having a person supply reward and/or punishment as an additional input to the reward function [5,10,13,14,31].…”

Bootstrapping intrinsically motivated learning with human demonstration

“…In such systems, the learner scarcely explores on his own to learn tasks or skills beyond what it has observed with a human. Many prior works have given a human trainer control of the reinforcement learning reward [8], [9], provide advice [10], or tele-operate the agent during training [11]. However, the more dependent on the human the system, the more challenging learning from interactions with a human is, due to limitations like human patience, ambiguous human input, correspondence problems [12] etc.…”

Cognitive and Behavioral Model Ensembles for Autonomous Virtual Characters

“…Cognitive models select goals and make deliberative decisions, in contrast to behavioral models which make only reactive decisions, and both modeling techniques are powerful tools for building autonomous characters. Though considered difficult to create, many models exist that perform specific tasks (Burke et al 2001), reach designated goals (Funge et al 1999), or behave in a stylistic manner (Blumberg et al 2002).…”