Towards Automatic Discovering of Deep Hybrid Network Architecture for Sequential Recommendation

Cheng, Mingyue; Liu, Zhiding; Liu, Qi; Ge, Shenyang; Chen, Enhong

doi:10.1145/3485447.3512066

Cited by 19 publications

(9 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The shallow architectures limit their capabilities significantly. To solve the problem, taking inspiration from recently proposed works [8], [24], we add a learnable parameter λ L initialed as 0 to each skip connection component. Specifically, for the SASRec [6] backbone contained by CANet, the transformer architecture enhanced with our simple residual modification can be expressed as…”

Section: B a Review On Deep Sequential Recommendation Modelsmentioning

confidence: 99%

“…To tackle such problem, embedding size searching methods [13], [14] are widely studied, aiming to search for suitable embedding sizes for input features in an automatic way to reduce parameters in embedding layer. But on the other hand, recent few studies point out that even the hidden layers of recommendation models can also be wider and deeper to gain a performance boost [8], [16]. How to achieve computation efficient in recommendation is becoming more and more important and some works have been devoted in this searching area.…”

Section: B Efficient Recommendationmentioning

confidence: 99%

“…A widely accepted view is that recommendation * Enhong Chen is the corresponding author. models are generally huge, and several recent efforts have been devoted to reveal that a larger network could yield obvious performance improvements in SR tasks [8], [9]. Nevertheless, we argue that such improvements are unaffordable in real recommender systems since larger models inevitably result in expensive computational cost and require higher-performance hardware.…”

mentioning

confidence: 92%

See 2 more Smart Citations

One Person, One Model--Learning Compound Router for Sequential Recommendation

Liu¹,

Cheng²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Section: B a Review On Deep Sequential Recommendation Modelsmentioning

confidence: 99%

Section: B Efficient Recommendationmentioning

confidence: 99%

mentioning

confidence: 92%

See 1 more Smart Citation

One Person, One Model--Learning Compound Router for Sequential Recommendation

Liu¹,

Cheng²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

“…Hence, researchers begin to explore more expressive feature maps for improving classification capacity. Deep learning-based models [6,17,22,24,39,49,50] have achieved remarkable success and have become ever-increasingly prevalent over past advancements. The main reason is that discriminative features related to time series can be learned in an end-to-end manner, which significantly saves manual feature engineering efforts.…”

Section: Introductionmentioning

confidence: 99%

“…The main reason is that discriminative features related to time series can be learned in an end-to-end manner, which significantly saves manual feature engineering efforts. Among them, convolutional-based methods nearly have become the dominant approach due to the strong representation capacity of convolution operations [6,31]. In general, the strengths of convolutional models in performing time series classification can be summarized as follows:…”

Section: Introductionmentioning

confidence: 99%

FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification

Cheng¹,

Liu²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep learning-based algorithms, e.g., convolutional networks, have significantly facilitated multivariate time series classification (MTSC) task. Nevertheless, they suffer from the limitation in modeling longrange dependence due to the nature of convolution operations.Recent advancements have shown the potential of transformers to capture long-range dependence. However, it would incur severe issues, such as fixed scale representations, temporal-invariant and quadratic time complexity, with transformers directly applicable to the MTSC task because of the distinct properties of time series data. To tackle these issues, we propose FormerTime, an hierarchical representation model for improving the classification capacity for the MTSC task. In the proposed FormerTime, we employ a hierarchical network architecture to perform multi-scale feature maps. Besides, a novel transformer encoder is further designed, in which an efficient temporal reduction attention layer and a well-informed contextual positional encoding generating strategy are developed. To sum up, FormerTime exhibits three aspects of merits: (1) learning hierarchical multi-scale representations from time series data, (2) inheriting the strength of both transformers and convolutional networks, and (3) tacking the efficiency challenges incurred by the self-attention mechanism. Extensive experiments performed on 10 publicly available datasets from UEA archive verify the superiorities of the FormerTime compared to previous competitive baselines. CCS CONCEPTS• Computing methodologies → Neural networks; • Mathematics of computing → Time series analysis.

show abstract

Multiscale spatial‐temporal transformer with consistency representation learning for multivariate time series classification

Wu,

Qiu,

Wang

et al. 2024

Concurrency and Computation

View full text Add to dashboard Cite

SummaryMultivariate time series classification holds significant importance in fields such as healthcare, energy management, and industrial manufacturing. Existing research focuses on capturing temporal changes or calculating time similarities to accomplish classification tasks. However, as the state of the system changes, capturing spatial‐temporal consistency within multivariate time series is key to the ability of the model to classify accurately. This paper proposes the MSTformer model, specifically designed for multivariate time series classification tasks. Based on the Transformer architecture, this model uniquely focuses on multiscale information across both time and feature dimensions. The encoder, through a designed learnable multiscale attention mechanism, divides data into sequences of varying temporal scales to learn multiscale temporal features. The decoder, which receives the spatial view of the data, utilizes a dynamic scale attention mechanism to learn spatial‐temporal consistency in a one‐dimensional space. In addition, this paper proposes an adaptive aggregation mechanism to synchronize and combine the outputs of the encoder and decoder. It also introduces a multiscale 2D separable convolution designed to learn spatial‐temporal consistency in two‐dimensional space, enhancing the ability of the model to learn spatial‐temporal consistency representation. Extensive experiments were conducted on 30 datasets, where the MSTformer outperformed other models with an average accuracy rate of 85.6%. Ablation studies further demonstrate the reliability and stability of MSTformer.

show abstract

Towards Automatic Discovering of Deep Hybrid Network Architecture for Sequential Recommendation

Cited by 19 publications

References 26 publications

One Person, One Model--Learning Compound Router for Sequential Recommendation

One Person, One Model--Learning Compound Router for Sequential Recommendation

FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification

Multiscale spatial‐temporal transformer with consistency representation learning for multivariate time series classification

Contact Info

Product

Resources

About