On-line system identification using context discernment

Holmstrom, Lars; Santiago, R.A.; Lendaris, George G.

doi:10.1109/ijcnn.2005.1555953

Cited by 7 publications

(8 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We propose, however, that context discernment can be addressed/enhanced by the accumulation of an intelligent agent's experience with its environment. Adaptive Critic Reinforcement Learning methods [3] have previously been applied to explore experience based context discernment in toy problems [4][5] [6]. While this paper does not explore the learning phase of the autonomous accumulation of experience, it does show how experience can be leveraged in a real world robotic application for the task of context discernment.…”

Section: Introductionmentioning

confidence: 99%

Experience Based Surface Discernment by a Quadruped Robot

Holmstrom

Toland

Lendaris

2007

2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing

View full text Add to dashboard Cite

Abstract-The task of autonomous surface discernment by an AIBO robotic dog is addressed. Different surface textures (plywood board, thin foam, short carpet, shag carpet) as well as different inclines (0 and 10 degrees) are considered. Using a genetic algorithm, gaits are designed which allow the robot to traverse each of these surfaces in an (approximately) optimal fashion. Frequency domain analysis of actuator readings from individual leg joints is performed for data collected using each gait on each surface type. It is found that the spectral content of these signals is significantly dependent on the characteristics of both the gait in use and the surface being walked upon. Using tap-delay Adaline neural networks to integrate actuator readings from 15 independent joints into a set of models of different gait/surface experiences, an algorithm is designed which uses these experiences to yield high classification rates across surface transitions and with low latency.

show abstract

Section: Introductionmentioning

confidence: 99%

Experience Based Surface Discernment by a Quadruped Robot

Holmstrom

Toland

Lendaris

2007

2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing

View full text Add to dashboard Cite

show abstract

“…A key to this approach is the notion of context discernment [8][13] [15]. A recent NWCIL result demonstrates the ability of a neuralnetwork based agent to efficiently discern which surface type, from among a set of candidates, a quadruped robot experiences in real-time.…”

Section: Introductionmentioning

confidence: 99%

Effectiveness of a Coupled Oscillator Network for Surface Discernment by a Quadruped Robot based on Kinesthetic Experience

Toland

Holstrom

Lendaris

2007

2007 International Joint Conference on Neural Networks

View full text Add to dashboard Cite

Abstract-Inspired by examples of oscillatory circuits in biological brains, we explore a hypothesis that one role of dynamical neural networks observed in biological sensory systems is to amplify subtle differences in sensory data, which in turn simplifies the task of classifying external stimuli. The authors recently developed a method for classifying the surface walked upon by a quadruped robotic dog [9]. The method developed utilizes time series data from the dog's joint sensors (kinesthetic vector). Employing the same data, an experiment was set up to explore the above hypothesis, comparing the relative accuracy of classifying (discerning) the surface type experienced by the robot, both with and without the inclusion of a system of coupled nonlinear oscillators in the data processing stream. These experiments demonstrated a significant increase in classification rate (on average) when the sensory data was passed through a coupled oscillator system to precondition the signals prior to inputting to a PNN type neural network classifier, in comparison with the result obtained by feeding the data to the PNN without preconditioning. From an implementation point of view, it is significant that these results were obtained via a coupled oscillator whose inter-oscillator weights were randomly instantiated. Some of the results are provided in terms of the Lyapunov Exponent and the spectral radius of the inter-oscillator weight matrix.

show abstract

“…This was a useful starting point to demonstrate that given such manifolds and corresponding mappings, a RL process could be configured to implement an HLLA to develop a context discerner with desired properties ("effective and efficient"). That is, it was demonstrated that a RL process could train an Agent to learn Component B of the Experience definition given in Section Ia ("to efficiently and effectively select a model from the repository as changes in context occur"); the particular HLLA employed so far is called Contextual Reinforcement Learning [6] [13].…”

Section: Iv3 Discussionmentioning

confidence: 99%

“…In the remainder of this Section, three examples of using a DHP AC implementation of the HLLA concept called Contextual Reinforcement Learning [6] demonstrate successful application of the notions put forth in this paper to the creation of an EBSID algorithm. Note: the HLLA of these examples is applied in a single level of the Context Space Hierarchy (subject of a future paper); this concept envisions specialized HLLAs for various levels of the Hierarchy.…”

Section: Iv2 Hlla To Create Eb-algorithm For Sid (Ebsid)mentioning

confidence: 95%

See 1 more Smart Citation

Experience-Based Identification and Control via Higher-Level Learning and Context Discernment

Lendaris¹

The 2006 IEEE International Joint Conference on Neural Network Proceedings

Self Cite

View full text Add to dashboard Cite

Abstract-In AI systems so far developed, more knowledge (typically stored as "rules") entails slower processing; in the case of humans, the more knowledge attained (in the form of experience), the speed/efficiency of performing new related tasks is improved. Experience-Based (EB) Identification and Control is explored with the objective of achieving more human-like processes for 'intelligent' computing Agents. The notion of experience is being successfully addressed via a novel concept for applying Reinforcement Learning (RL), called HLLA -Higher Level Learning Algorithm. The key idea is to re-purpose the RL method (to a "higher level") such that instead of creating an optimal controller for a given task, an already achieved collection of such solutions for a variety of related contexts is provided (as an experience repository), and HLLA creates a strategy for optimally selecting a solution from the repository. The selection process is triggered by the Agent becoming aware that a change in context has occurred, followed by the Agent seeking information about what changeda process here called context discernment -and finally, by selection. Typically, context discernment entails a form of system identification (SID); substantial enhancement of SID is also achieved via the EB methods. Examples are given.

show abstract

On-line system identification using context discernment

Cited by 7 publications

References 3 publications

Experience Based Surface Discernment by a Quadruped Robot

Experience Based Surface Discernment by a Quadruped Robot

Effectiveness of a Coupled Oscillator Network for Surface Discernment by a Quadruped Robot based on Kinesthetic Experience

Experience-Based Identification and Control via Higher-Level Learning and Context Discernment

Contact Info

Product

Resources

About