Empirical analysis of an on-line adaptive system using a mixture of Bayesian networks

Kitakoshi, Daisuke; Shioya, Hiroyuki; Nakano, Ryoko

doi:10.1016/j.ins.2010.04.001

Cited by 15 publications

(4 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A Bayesian Network (BN) is known as stochastic knowledge representation model and is applied to a variety of problem domains, such as data mining, prediction, and so on [1], [4]. Its structure is represented as directed acyclic graph expressing the dependences among random variables 1 , 2 , .…”

Section: A Bayesian Networkmentioning

confidence: 99%

See 1 more Smart Citation

A probabilistic reasoning algorithm for Bayesian networks by simplifying their structures

Kitakoshi

Shuhei

Suzuki

2011

2011 IEEE International Conference on Granular Computing

Self Cite

View full text Add to dashboard Cite

This article describes a new probabilistic reasoning algorithm for Bayesian networks, one of the stochastic models. The proposed method, called Extended LBPC, takes advantage of existing reasoning methods and statistical techniques such as hypothesis testing and interval estimate.Several computer simulations demonstrate that the proposed algorithm can perform appropriate and adaptable probabilistic inference depending on the complexity of problems in terms of both accuracy and computational time, compared to other conventional methods.

show abstract

Section: A Bayesian Networkmentioning

confidence: 99%

“…Adoptingˆas the reasoning result if it is within the confidence interval; (4) Performing hypothesis test ifˆis not within the interval; (5) As a result of (4),…”

Section: B Extended Lbpcmentioning

confidence: 99%

A probabilistic reasoning algorithm for Bayesian networks by simplifying their structures

Kitakoshi

Shuhei

Suzuki

2011

2011 IEEE International Conference on Granular Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Normally, RL agents initialize the policies when they are placed in a new environment and the learning process starts afresh each time. Effective adjustment to an unknown environment becomes possible by using statistical methods, such as a Bayesian network model [4,5], mixture probability and clustering distribution [6], etc., which consist of observational data on multiple environments that the agents have learned in the past [7,8]. However, the use of a mixture model of Bayesian networks increases the system's calculation time.…”

Section: Introductionmentioning

confidence: 99%

A Policy-Improving System for Adaptability to Dynamic Environments Using Mixture Probability and Clustering Distribution

Phommasak¹,

Kitakoshi²,

Mao³

et al. 2014

JCC

Self Cite

View full text Add to dashboard Cite

Along with the increasing need for rescue robots in disasters such as earthquakes and tsunami, there is an urgent need to develop robotics software for learning and adapting to any environment. A reinforcement learning (RL) system that improves agents' policies for dynamic environments by using a mixture model of Bayesian networks has been proposed, and is effective in quickly adapting to a changing environment. However, the increase in computational complexity requires the use of a high-performance computer for simulated experiments and in the case of limited calculation resources, it becomes necessary to control the computational complexity. In this study, we used an RL profit-sharing method for the agent to learn its policy, and introduced a mixture probability into the RL system to recognize changes in the environment and appropriately improve the agent's policy to adjust to a changing environment. We also introduced a clustering distribution that enables a smaller, suitable selection, while maintaining a variety of mixture probability elements in order to reduce the computational complexity and simultaneously maintain the system's performance. Using our proposed system, the agent successfully learned the policy and efficiently adjusted to the changing environment. Finally, control of the computational complexity was effective, and the decline in effectiveness of the policy improvement was controlled by using our proposed system.

show abstract

“…Normally, RL agents need to initialize the policies when they are placed in a new environment and the learning process starts afresh each time. Effective adjustment to an unknown environment becomes possible by using statistical methods, such as a Bayesian network model [5] [6], mixture probability and clustering distribution [7] [8], etc., which consist of observational data on multiple environments that the agents have learned in the past [9] [10]. However, the use of a mixture model of Bayesian networks increases the system's calculation time.…”

mentioning

confidence: 99%

A Reinforcement Learning System to Dynamic Movement and Multi-Layer Environments

Phommasak¹,

Kitakoshi²,

Shioya³

et al. 2014

JILSA

Self Cite

View full text Add to dashboard Cite

There are many proposed policy-improving systems of Reinforcement Learning (RL) agents which are effective in quickly adapting to environmental change by using many statistical methods, such as mixture model of Bayesian Networks, Mixture Probability and Clustering Distribution, etc. However such methods give rise to the increase of the computational complexity. For another method, the adaptation performance to more complex environments such as multi-layer environments is required. In this study, we used profit-sharing method for the agent to learn its policy, and added a mixture probability into the RL system to recognize changes in the environment and appropriately improve the agent's policy to adjust to the changing environment. We also introduced a clustering that enables a smaller, suitable selection in order to reduce the computational complexity and simultaneously maintain the system's performance. The results of experiments presented that the agent successfully learned the policy and efficiently adjusted to the changing in multi-layer environment. Finally, the computational complexity and the decline in effectiveness of the policy improvement were controlled by using our proposed system.

show abstract

Empirical analysis of an on-line adaptive system using a mixture of Bayesian networks

Cited by 15 publications

References 46 publications

A probabilistic reasoning algorithm for Bayesian networks by simplifying their structures

A probabilistic reasoning algorithm for Bayesian networks by simplifying their structures

A Policy-Improving System for Adaptability to Dynamic Environments Using Mixture Probability and Clustering Distribution

A Reinforcement Learning System to Dynamic Movement and Multi-Layer Environments

Contact Info

Product

Resources

About