Learning to Reproduce Fluctuating Time Series by Inferring Their Time-Dependent Stochastic Properties: Application in Robot Learning Via Tutoring

Murata, Satoshi; Namikawa, Jun; Arie, Hiroaki; Sugano, Shigeki; Tani, Jun

doi:10.1109/tamd.2013.2258019

Cited by 73 publications

(60 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although we utilized MTRNN in this paper, we believe that the method can also be applied to works using other dynamical models, such as S-CTRNN to generate robot motion by predicting also the variance of the robot's motion. [18] Further on, we plan to implement the method to other developmental robotics studies to achieve a more sophisticated system based on human development.…”

Section: Discussionmentioning

confidence: 99%

Preferential training of neurodynamical model based on predictability of target dynamics

Nishide

Nobuta²,

Okuno

et al. 2015

Advanced Robotics

View full text Add to dashboard Cite

Intrinsic motivation is one of the keys in implementing the mechanism of interest to robots. In this paper, we present a method to apply intrinsic motivation in dynamics learning with predictable and unpredictable targets in view. The robot's arm is used for the predictable target and the human's arm is used for the unpredictable target in the experiment. The learning algorithm based on intrinsic motivation will automatically set a larger weight to targets that would contribute to decreasing the training error, while setting a smaller weight to others. A neurodynamical model, namely multiple timescales recurrent neural network (MTRNN), is utilized for studying the robot's arm/external object dynamics. Training of MTRNN is done using the back propagation through time (BPTT) algorithm. We modify the BPTT algorithm by the following two steps. (1) Evaluate predictability of robot's arm/objects using training error of MTRNN.(2) Assign a preference ratio, which represents the weight of the training, to each object based on predictability. The proposed training method would focus more on reducing training error of predictable objects compared to normal BPTT, where training error is equally treated for every object. Experiments were conducted using an actual robot platform, moving the robot's arm while a human moves his arm in the robot's camera view. The results of the experiment showed that the proposed training method could achieve smaller training error of the robot's arm visuomotor dynamics, which is predictable from the robot's motor command, compared to general training with BPTT. Evaluation of MTRNN as a forward model to predict untrained data showed that the proposed model is capable of predicting the robot's hand motion, specifically with larger number of nodes.

show abstract

Section: Discussionmentioning

confidence: 99%

Preferential training of neurodynamical model based on predictability of target dynamics

Nishide

Nobuta²,

Okuno

et al. 2015

Advanced Robotics

View full text Add to dashboard Cite

show abstract

“…The gradients ∂L(θ) ∂θ can be obtained by the conventional back-propagation through time (BPTT) method [23]. Details of the calculation of gradients are provided in [22].…”

Section: B Predictive Learning Methodsmentioning

confidence: 99%

“…Feldman and Friston [21] suggested that in this scheme, attention can be understood as the process of inferring the level of uncertainty or precision of probabilistic representations of the world. Recently, Murata et al [22] proposed a new RNN-based model called a stochastic continuous-time RNN (S-CTRNN). The S-CTRNN can learn to generate predictions about both sensory inputs and their uncertainty (inverse precision) in terms of variance by minimizing a negative log-likelihood consisting of log-uncertainty and precision-weighted prediction error.…”

Section: Introductionmentioning

confidence: 99%

Predictive learning with uncertainty estimation for modeling infants' cognitive development with caregivers: A neurorobotics experiment

Murata

Tomioka

Nakajo

et al. 2015

2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob)

Self Cite

View full text Add to dashboard Cite

Dynamic interactions with caregivers are essential for infants to develop cognitive abilities, including aspects of action, perception, and attention. We hypothesized that these abilities can be acquired through the predictive learning of sensory inputs including their uncertainty (inverse precision) in terms of variance. To examine our hypothesis from the perspective of cognitive developmental robotics, we conducted a neurorobotics experiment involving a ball-playing interaction task between a human experimenter representing a caregiver and a small humanoid robot representing an infant. The robot was equipped with a dynamic generative model called a stochastic continuous-time recurrent neural network (S-CTRNN). The S-CTRNN learned to generate predictions about both the visuoproprioceptive states of the robot and the uncertainty of these states by minimizing a negative log-likelihood consisting of loguncertainty and precision-weighted prediction error. The experimental results showed that predictive learning with uncertainty estimation enabled the robot to acquire infant-like cognitive abilities through dynamic interactions with the experimenter. We also discuss the effects of infant-directed modifications observed in caregiver-infant interactions on the development of these abilities.

show abstract

“…The second factor that proves the appropriateness of B-SRGP (non-gn) for INS/GPS applications is based on the consideration of measurement noise as non-Gaussian distributed. Indeed, setting the shape parameter as greater than zero, that is, = 2, reduces the prediction error because the estimated noise variance and therefore the covariance matrix which constitutes the likelihood functions become small [32]. These two main factors contribute to increasing the position prediction accuracy.…”

Section: (A) Determination Of the Partial And Full Gps Outagesmentioning

confidence: 99%

A Novel Probabilistic Approach for Vehicle Position Prediction in Free, Partial, and Full GPS Outages

Havyarimana

Wang

Xiao

2015

Mathematical Problems in Engineering

View full text Add to dashboard Cite

In this paper, a novel framework is developed with the intention of continuously predicting vehicle position even in the challenging environments such as partial and full GPS outages. To achieve this, the Bayesian-Sparse Random Gaussian Prediction (B-SRGP) approach is proposed where the sparse random Gaussian matrix which obeys the restricted isometry property with high probability is adopted to handle the measurement model. During the full GPS outages, the proposed method fuses all available INS measurements to improve the vehicle position prediction whereas in free outages only the GPS data are processed. Besides, the Bayesian inference is used to specifically deal with the vehicle position prediction in partial GPS outages where data from both GPS and INS are taken as inputs. In all cases, measurement noises are assumed to be non-Gaussian distributed and follow the generalized error distribution. The performance of B-SRGP is evaluated with respect to real-world data collected using Smartphonebased vehicular sensing model. The proposed method is tested when measurement noises are both Gaussian and non-Gaussian distributed and also compared with the existing prediction methods. Experimental results confirm that B-SRGP presents higher accuracy prediction and lower mean-squared prediction error for vehicle position when measurement noises are non-Gaussian distributed.

show abstract

Learning to Reproduce Fluctuating Time Series by Inferring Their Time-Dependent Stochastic Properties: Application in Robot Learning Via Tutoring

Cited by 73 publications

References 22 publications

Preferential training of neurodynamical model based on predictability of target dynamics

Preferential training of neurodynamical model based on predictability of target dynamics

Predictive learning with uncertainty estimation for modeling infants' cognitive development with caregivers: A neurorobotics experiment

A Novel Probabilistic Approach for Vehicle Position Prediction in Free, Partial, and Full GPS Outages

Contact Info

Product

Resources

About