Human Representation Learning

Radulescu, Angela; Shin, Yeon Soon; Niv, Yaron

doi:10.1146/annurev-neuro-092920-120559

Cited by 44 publications

(36 citation statements)

References 161 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That is, the context of a few previous events is a pointer to responses made in the same context, stored on the long-term. Since the α and λ parameters are specific to the hierarchy levels and are inferred from the data, the sequence learning problem is cast as learning a prior over ’What should I remember?’, as in to other approaches to representation learning [27].…”

Section: Methodsmentioning

confidence: 99%

Tracking human skill learning with a hierarchical Bayesian sequence model

Éltető

Németh

Janacsek

et al. 2022

Preprint

View full text Add to dashboard Cite

Humans can implicitly learn complex perceptuo-motor skills over the course of large numbers of trials. This likely depends on our becoming better able to take advantage of ever richer and temporally deeper predictive relationships in the environment. Here, we offer a novel characterization of this process, fitting a non-parametric, hierarchical Bayesian sequence model to the reaction times of human participants' responses over ten sessions, each comprising thousands of trials, in a serial reaction time task involving higher-order dependencies. The model, adapted from the domain of language, forgetfully updates trial-by-trial, and seamlessly combines predictive information from shorter and longer windows onto past events, weighing the windows proportionally to their predictive power. As the model defines a prior over window depths, we were able to determine the extent to which the internal predictions of individual participant depended on how many previous elements. Already in the first session, the model showed that participants had begun to rely on two previous elements (i.e., trigrams), thereby successfully adapting to the most prominent higher-order structure in the task. The extent to which local statistical fluctuations influenced participants' responses waned over subsequent sessions, as subjects forgot the trigrams less and evidenced skilled performance. By the eighth session, a subset of participants shifted their prior further to consider a context deeper than two previous elements. Finally, participants showed resistance to interference and slow forgetting of the old sequence when it was changed in the final sessions. Model parameters for individual subjects covaried appropriately with independent measures of working memory. In sum, the model offers the first principled account of the adaptive complexity and nuanced dynamics of humans' internal sequence representations during long-term implicit skill learning.

show abstract

Section: Methodsmentioning

confidence: 99%

Tracking human skill learning with a hierarchical Bayesian sequence model

Éltető

Németh

Janacsek

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…6, our algorithm significantly outperforms the other two, approaching the oracle that makes correct choice at every round. This experiment suggests that the ability to learn representations and shift attention [2,41] to adapt to environment changes plays a crucial role in learning efficiently.…”

Section: Illustrative Examplesmentioning

confidence: 99%

“…Humans are naturally endowed with the ability to learn and transfer experience to later unseen tasks. The key mechanism enabling such versatility is the abstraction of past experience into a 'basis set' of simpler representations that can be used to construct new strategies much more efficiently in future complex environments [1,2].…”

Section: Introductionmentioning

confidence: 99%

Non-Stationary Representation Learning in Sequential Linear Bandits

Qin¹,

Menara²,

Oymak³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we study representation learning for multi-task decision-making in non-stationary environments. We consider the framework of sequential linear bandits, where the agent performs a series of tasks drawn from distinct sets associated with different environments. The embeddings of tasks in each set share a low-dimensional feature extractor called representation, and representations are different across sets. We propose an online algorithm that facilitates efficient decision-making by learning and transferring non-stationary representations in an adaptive fashion. We prove that our algorithm significantly outperforms the existing ones that treat tasks independently. We also conduct experiments using both synthetic and real data to validate our theoretical insights and demonstrate the efficacy of our algorithm.

show abstract

“…Yet, human beings have shown the remarkable ability to make good decisions in such environments. At the core of this ability is the flexibility to adapt their behaviors in different situations [1]. Such adaption is usually fast since humans learn to abstract experiences into compact representations that support the efficient construction of new strategies [2].…”

Section: Introductionmentioning

confidence: 99%

Representation Learning for Context-Dependent Decision-Making

Qin¹,

Menara²,

Oymak³

et al. 2022

Preprint

View full text Add to dashboard Cite

Humans are capable of adjusting to changing environments flexibly and quickly. Empirical evidence has revealed that representation learning plays a crucial role in endowing humans with such a capability. Inspired by this observation, we study representation learning in the sequential decision-making scenario with contextual changes. We propose an online algorithm that is able to learn and transfer contextdependent representations and show that it significantly outperforms the existing ones that do not learn representations adaptively. As a case study, we apply our algorithm to the Wisconsin Card Sorting Task, a well-established test for the mental flexibility of humans in sequential decision-making. By comparing our algorithm with the standard Q-learning and Deep-Q learning algorithms, we demonstrate the benefits of adaptive representation learning. Y. Qin and F. Pasqualetti are with the Department of Mechanical Engineering (

show abstract

Human Representation Learning

Cited by 44 publications

References 161 publications

Tracking human skill learning with a hierarchical Bayesian sequence model

Tracking human skill learning with a hierarchical Bayesian sequence model

Non-Stationary Representation Learning in Sequential Linear Bandits

Representation Learning for Context-Dependent Decision-Making

Contact Info

Product

Resources

About