XCSF with tile coding in discontinuous action-value landscapes

Lanzi, Pier Luca; Loiacono, Daniele

doi:10.1007/s12065-015-0129-7

Cited by 3 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Description of arm-reaching movements model. To model arm-reaching movements we used a kinematic model of the arm muscles 97,98 . The arm kinematics were modeled in the transverse plane by analyzing the effect of six muscles on the arm dynamics, cf.…”

Section: Methodsmentioning

confidence: 99%

Predictive learning as a network mechanism for extracting low-dimensional latent space representations

et al. 2021

View full text Add to dashboard Cite

Artificial neural networks have recently achieved many successes in solving sequential processing and planning tasks. Their success is often ascribed to the emergence of the task’s low-dimensional latent structure in the network activity – i.e., in the learned neural representations. Here, we investigate the hypothesis that a means for generating representations with easily accessed low-dimensional latent structure, possibly reflecting an underlying semantic organization, is through learning to predict observations about the world. Specifically, we ask whether and when network mechanisms for sensory prediction coincide with those for extracting the underlying latent variables. Using a recurrent neural network model trained to predict a sequence of observations we show that network dynamics exhibit low-dimensional but nonlinearly transformed representations of sensory inputs that map the latent structure of the sensory environment. We quantify these results using nonlinear measures of intrinsic dimensionality and linear decodability of latent variables, and provide mathematical arguments for why such useful predictive representations emerge. We focus throughout on how our results can aid the analysis and interpretation of experimental data.

show abstract

Section: Methodsmentioning

confidence: 99%

Predictive learning as a network mechanism for extracting low-dimensional latent space representations

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Description of arm-reaching movements model. To model arm-reaching movements we used a kinematic model of the arm muscles (84,85). The arm kinematics were modeled in the transverse plane by analyzing the effect of 6 muscles on the arm dynamics, cf.…”

mentioning

confidence: 99%

Predictive learning as a network mechanism for extracting low-dimensional latent space representations

Recanatesi

Farrell

Lajoie

et al. 2018

Preprint

View full text Add to dashboard Cite

Neural networks have achieved many recent successes in solving sequential processing and planning tasks. Their success is often ascribed to the emergence of the task's low-dimensional latent structure in the network activity -i.e., in the learned neural representations. Similarly, biological neural circuits and in particular the hippocampus may produce representations that organize semantically related episodes. Here, we investigate the hypothesis that representations with low-dimensional latent structure, reflecting such semantic organization, result from learning to predict observations about the world. Specifically, we ask whether and when network mechanisms for sensory prediction coincide with those for extracting the underlying latent variables. Using a recurrent neural network model trained to predict a sequence of observations in a simulated spatial navigation task, we show that network dynamics exhibit low-dimensional but nonlinearly transformed representations of sensory inputs that capture the latent structure of the sensory environment. We quantify these results using nonlinear measures of intrinsic dimensionality which highlight the importance of the predictive aspect of neural representations, and provide mathematical arguments for when and why these representations emerge. We focus throughout on how our results can aid the analysis and interpretation of experimental data.

show abstract

“…Much research has been devoted to piecewise function approximation with a variant called XCSF [17], in which each rule computes a realvalued prediction as a function f of its input x: ''IF this rule applies to input x THEN output f ðxÞ''. Lanzi and Loiacono's work in this issue [8] is a good example of XCS's flexibility as it replaces the earlier functions used in XCSF with tile coding. Although tile coding is a popular function approximation method in RL its success depends on its parameterisation, and tile coding itself is not adaptive.…”

mentioning

confidence: 99%