Slow Down to Go Better: A Survey on Slow Feature Analysis

Song, Pengyu; Zhao, Chunhui

doi:10.1109/tnnls.2022.3201621

Cited by 72 publications

(8 citation statements)

References 117 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, we confirm their results numerically by using the potential depicted in Fig. 2c , but we also propose that the modified TAE can be extended to slow feature analysis (SFA) (Wiskott and Sejnowski, 2002 ; Berkes and Wiskott, 2005 ; Song and Zhao, 2022 ) (see Fig. 1b ), which can render the latent variables orthogonal in multidimensional cases, since Eq.…”

Section: Resultssupporting

confidence: 79%

Chasing collective variables using temporal data-driven strategies

Chen

Chipot

2023

QRB Discovery

View full text Add to dashboard Cite

The convergence of free-energy calculations based on importance sampling depends heavily on the choice of collective variables (CVs), which in principle, should include the slow degrees of freedom (DOFs) of the biological processes to be investigated. Autoencoders (AEs), as emerging data-driven dimension reduction tools, have been utilized for discovering CVs. AEs, however, are often treated as black boxes, and what AEs actually encode during training, and whether the latent variables from encoders are suitable as CVs for further free-energy calculations remains unknown. In this contribution, we review AEs and their time-series-based variants, including time-lagged AEs (TAEs) and modified TAEs, as well as the closely related model variational approach for Markov processes networks, or VAMPnets. We then show through numerical examples that AEs learn the high-variance modes instead of the slow modes. In stark contrast, time series-based models are able to capture the slow modes. Moreover, both modified TAEs with extensions from slow feature analysis (SFA) and the state-free reversible VAMPnets (SRVs) can yield orthogonal multidimensional CVs. As an illustration, we employ SRVs to discover the CVs of the isomerizations of N-acetyl-N'-methylalanylamide (NANMA) and trialanine by iterative learning with trajectories from biased simulations. Last, through numerical experiments with anisotropic diffusion, we investigate the potential relationship of time-series-based models and committor probabilities.

show abstract

Section: Resultssupporting

confidence: 79%

Chasing collective variables using temporal data-driven strategies

Chen

Chipot

2023

QRB Discovery

View full text Add to dashboard Cite

show abstract

“…The expert prior knowledge comes from the expert diagnostic experience, which can be generalized to fault diagnosis tasks under various working conditions [18]. Research shows that the expert prior knowledge enables the model to learn more generalized fault features and improve the diagnostic accuracy of the model [18][19][20][21][22][23]. However, the expert prior knowledge is often limited and easily disturbed by noise.…”

Section: Introductionmentioning

confidence: 99%

Knowledge-driven domain adaptation strategy for rotating machinery fault diagnosis under varying working condition

Chang,

Yao,

Chen

et al. 2024

Meas. Sci. Technol.

Self Cite

View full text Add to dashboard Cite

Due to the frequent switch of the working condition, fault diagnosis model for rotating machinery established on the training set (the source domain) cannot be effectively applied to the test set (the target domain). Traditional domain adaptation methods address this issue by performing feature alignment between the source and target domain, which ignores the positive guidance provided by prior knowledge. In this paper, we propose a prior knowledge-driven domain adaptation (PKDA) method for varying working condition fault diagnosis of rotating machinery, where a self-supervised learning framework is designed to integrate expert prior knowledge and structural prior knowledge. First, an expert prior knowledge guidance module is designed to extract features with physical significance. Then we align the features between the source and target domain by the max mean discrepancy (MMD) metric. Besides, a progressive Shannon entropy minimization strategy is proposed to realize the feature distribution structure of intra-class compression and inter-class separation, which can effectively integrate the structural prior knowledge. In this way, PKDA can effectively utilize the prior knowledge to achieve better performance in varying working condition fault diagnosis tasks. The effectiveness of the proposed method is illustrated by an open-source rolling bearing fault dataset from Case Western Reserve University and an open-source gearbox fault dataset from Southeast University.

show abstract

“…A well-known implementation of this prior is Slow Feature Analysis (SFA), an unsupervised learning algorithm that reduces the dimensionality of its input by identifying slowly changing dimensions in the data [31, 39, 40]. SFA first isolates independent components in the data and then extracts those components that change slowly, under the premise that slower features are more meaningful representations of the data [31].…”

Section: Introductionmentioning

confidence: 99%

An inductive bias for slowly changing features in human reinforcement learning

Hedrich,

Schulz,

Hall-McMaster

et al. 2024

Preprint

View full text Add to dashboard Cite

Identifying goal-relevant features in novel environments is a central challenge for efficient behaviour. We asked whether humans address this challenge by relying on prior knowledge about common properties of reward-predicting features. One such property is the rate of change of features, given that behaviourally relevant processes tend to change on a slower timescale than noise. Hence, we asked whether humans are biased to learn more when task-relevant features are slow rather than fast. To test this idea, 100 human participants were asked to learn the rewards of two-dimensional bandits when either a slowly or quickly changing feature of the bandit predicted reward. Participants accrued more reward and achieved better generalisation to unseen feature values when a bandit's relevant feature changed slowly, and its irrelevant feature quickly, as compared to the opposite. Participants were also more likely to incorrectly base their choices on the irrelevant feature when it changed slowly versus quickly. These effects were stronger when participants experienced the feature speed before learning about rewards. Modelling this behaviour with a set of four function approximation Kalman filter models that embodied alternative hypotheses about how feature speed could affect learning revealed that participants had a higher learning rate for the slow feature, and adjusted their learning to both the relevance and the speed of feature changes. The larger the improvement in participants' performance for slow compared to fast bandits, the more strongly they adjusted their learning rates. These results provide evidence that human reinforcement learning favours slower features, suggesting a bias in how humans approach reward learning.

show abstract

Slow Down to Go Better: A Survey on Slow Feature Analysis

Cited by 72 publications

References 117 publications

Chasing collective variables using temporal data-driven strategies

Chasing collective variables using temporal data-driven strategies

Knowledge-driven domain adaptation strategy for rotating machinery fault diagnosis under varying working condition

An inductive bias for slowly changing features in human reinforcement learning

Contact Info

Product

Resources

About