Hierarchical clustering optimizes the tradeoff between compositionality and expressivity of task structures for flexible reinforcement learning

Liu, Rex G; Frank, Michael J.

doi:10.1016/j.artint.2022.103770

Cited by 4 publications

(4 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Human behavior in the real world is characterized by long-range dependencies between action plans and outcomes [5,6]. Much of this information is high-dimensional and complex, but can be compressed to yield more efficient, generalizable representations [9,10,[45][46][47][48]. Although how humans learn new behaviors according to such principles has remained elusive, previous work has shown that the computational burden involved in naturalistic behavior can be alleviated by setting "subgoals" that constrain planning over low-level actions [11,[13][14][15]37,49].…”

Section: Discussionmentioning

confidence: 99%

The successor representation subserves hierarchical abstraction for goal-directed behavior

Wientjes,

Holroyd

2024

PLoS Comput Biol

View full text Add to dashboard Cite

Humans have the ability to craft abstract, temporally extended and hierarchically organized plans. For instance, when considering how to make spaghetti for dinner, we typically concern ourselves with useful “subgoals” in the task, such as cutting onions, boiling pasta, and cooking a sauce, rather than particulars such as how many cuts to make to the onion, or exactly which muscles to contract. A core question is how such decomposition of a more abstract task into logical subtasks happens in the first place. Previous research has shown that humans are sensitive to a form of higher-order statistical learning named “community structure”. Community structure is a common feature of abstract tasks characterized by a logical ordering of subtasks. This structure can be captured by a model where humans learn predictions of upcoming events multiple steps into the future, discounting predictions of events further away in time. One such model is the “successor representation”, which has been argued to be useful for hierarchical abstraction. As of yet, no study has convincingly shown that this hierarchical abstraction can be put to use for goal-directed behavior. Here, we investigate whether participants utilize learned community structure to craft hierarchically informed action plans for goal-directed behavior. Participants were asked to search for paintings in a virtual museum, where the paintings were grouped together in “wings” representing community structure in the museum. We find that participants’ choices accord with the hierarchical structure of the museum and that their response times are best predicted by a successor representation. The degree to which the response times reflect the community structure of the museum correlates with several measures of performance, including the ability to craft temporally abstract action plans. These results suggest that successor representation learning subserves hierarchical abstractions relevant for goal-directed behavior.

show abstract

Section: Discussionmentioning

confidence: 99%

The successor representation subserves hierarchical abstraction for goal-directed behavior

Wientjes,

Holroyd

2024

PLoS Comput Biol

View full text Add to dashboard Cite

show abstract

“…Human behavior in the real world is characterized by long-range dependencies between action plans and outcomes (5,6). Much of this information is high-dimensional and complex, but can be compressed to yield more efficient, generalizable representations (9,10,(45)(46)(47)(48). Although how humans learn new behaviors according to such principles has remained elusive, previous work has shown that the computational burden involved in naturalistic behavior can be alleviated by setting "subgoals" that constrain planning over low-level actions (11,(13)(14)(15)37,49).…”

Section: Discussionmentioning

confidence: 99%

The successor representation subserves hierarchical abstraction for goal-directed behavior

Wientjes

Holroyd

2023

Preprint

View full text Add to dashboard Cite

Humans have the ability to craft abstract, temporally extended and hierarchically organized plans. For instance, when considering how to make spaghetti for dinner, we typically concern ourselves with useful "subgoals" in the task, such as cutting onions, boiling pasta, and cooking a sauce, rather than particulars such as how many cuts to make to the onion, or exactly which muscles to contract. A core question is how such decomposition of a more abstract task into logical subtasks happens in the first place. Previous research has shown that humans are sensitive to a form of hierarchical statistical learning named "community structure". Community structure is a common feature of abstract tasks characterized by a logical ordering of subtasks. This structure can be captured by a model where humans learn predictions of upcoming events multiple steps into the future, discounting predictions of events further away in time. One such model is the "successor representation", which has been argued to be useful for hierarchical abstraction. As of yet, no study has convincingly shown that this hierarchical abstraction can be put to use for goal-directed behavior. Here, we investigate whether participants utilize learned community structure to craft hierarchically informed action plans for goal-directed behavior. Participants were asked to search for paintings in a virtual museum, where the paintings were grouped together in "wings" representing community structure in the museum. We find that participants' choices accord with the hierarchical structure of the museum and that their response times are best predicted by a successor representation. The degree to which the response times reflect the community structure of the museum correlates with several measures of performance, including the ability to craft temporally abstract action plans. These results suggest that successor representation learning subserves hierarchical abstractions relevant for goal-directed behavior.

show abstract

“…Thus, in this study, the authors included kCC in their experimental investigation. On the other hand, some researchers focused on document clustering (Steinbach et al, 2000;Liu & Frank, 2022) and represented features in the term frequency-inverse document frequency (TF-IDF) matrix. Steinbach et al (2000) published a technical study on the behavior of clustering algorithms in the clustering of IR datasets.…”

Section: Related Workmentioning

confidence: 99%

“…On the other side, without knowing the exact number of clusters (k), hierarchical algorithms are used to draw the hierarchy of the clusters (Kuwil et al, 2020;Liu & Frank, 2022;Zhang et al, 2015;Zhang P. et al, 2019;Zhang W. et al, 2019). The majority of these algorithms, including minimumspanning tree-based clustering algorithms (MSTs), are suitable for massive, contemporary datasets with high dimensions (Wang et al, 2013).…”

Section: Introductionmentioning

confidence: 99%

Neighboring-Aware Hierarchical Clustering

Amer,

Al-Razgan,

Abdalla

et al. 2024

International Journal on Semantic Web and Information Systems

View full text Add to dashboard Cite

In this work, a simple yet robust neighboring-aware hierarchical-based clustering approach (NHC) is developed. NHC employs its dynamic technique to take into account the surroundings of each point when clustering, making it extremely competitive. NHC offers a straightforward design and reliable clustering. It comprises two key techniques, namely, neighboring- aware and filtering and merging. While the proposed neighboring-aware technique helps find the most coherent clusters, filtering and merging help reach the desired number of clusters during the clustering process. The NHC's performance, which includes all evaluation metrics and run time, has been thoroughly tested against nine clustering rivals using four similarity measures on several real-world numerical and textual datasets. The evaluation is done in two phases. First, we compare NHC to three common clustering methods and show its efficacy through empirical analysis. Second, a comparison with six relevant, contemporary competitors highlights NHC's extremely competitive performance.

show abstract

Hierarchical clustering optimizes the tradeoff between compositionality and expressivity of task structures for flexible reinforcement learning

Cited by 4 publications

References 46 publications

The successor representation subserves hierarchical abstraction for goal-directed behavior

The successor representation subserves hierarchical abstraction for goal-directed behavior

The successor representation subserves hierarchical abstraction for goal-directed behavior

Neighboring-Aware Hierarchical Clustering

Contact Info

Product

Resources

About