SAITS: Self-attention-based imputation for time series

Du, Wenjie; Côté, D.; Liu, Yan

doi:10.1016/j.eswa.2023.119619

Cited by 109 publications

(23 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We motivate the choice of DMSA against the original mechanism in the ablation study conducted in Section 4.4. Note that this type of diagonal attention mask has already proven better performance against the original one in Vision Transformer [31] to improve the overall performance of the model when dealing with small-size dataset, and more recently for time series imputation [18].…”

Section: Embedding Blockmentioning

confidence: 99%

ADF & TransApp: A Transformer-Based Framework for Appliance Detection Using Smart Meter Consumption Series

Petralia,

Charpentier,

Palpanas

2023

Proc. VLDB Endow.

View full text Add to dashboard Cite

Over the past decade, millions of smart meters have been installed by electricity suppliers worldwide, allowing them to collect a large amount of electricity consumption data, albeit sampled at a low frequency (one point every 30min). One of the important challenges these suppliers face is how to utilize these data to detect the presence/absence of different appliances in the customers' households. This valuable information can help them provide personalized offers and recommendations to help customers towards the energy transition. Appliance detection can be cast as a time series classification problem. However, the large amount of data combined with the long and variable length of the consumption series pose challenges when training a classifier. In this paper, we propose ADF, a framework that uses subsequences of a client consumption series to detect the presence/absence of appliances. We also introduce TransApp, a Transformer-based time series classifier that is first pretrained in a self-supervised way to enhance its performance on appliance detection tasks. We test our approach on two real datasets, including a publicly available one. The experimental results with two large real datasets show that the proposed approach outperforms current solutions, including state-of-the-art time series classifiers applied to appliance detection.

show abstract

Section: Embedding Blockmentioning

confidence: 99%

ADF & TransApp: A Transformer-Based Framework for Appliance Detection Using Smart Meter Consumption Series

Petralia,

Charpentier,

Palpanas

2023

Proc. VLDB Endow.

View full text Add to dashboard Cite

show abstract

“…This problem is alleviated by the LSTM Gers et al [15] which incorporates long-term stable memory over time using a series of gating functions. LSTM has been widely used, achieving state-ofthe-art results in numerous sequence learning applications, such as such as COVID-19 detection Hassan et al [16]; Li et al [17], video sequence processing Kong et al [18]; Donahue et al [19], cancer metastasis detection Kong et al [20], and time series analysis Du et al [21]. However, the traditional LSTM is not suitable for image sequence analysis since it uses fully-connected structure during both the input-to-state and state-to-state transitions, neglecting the spatial information.…”

Section: Related Workmentioning

confidence: 99%

Attention-driven tree-structured convolutional LSTM for high dimensional data understanding

Lu¹,

Kong²,

Gao³

et al. 2023

Front. Phys.

View full text Add to dashboard Cite

Modeling sequential information for image sequences is a vital step of various vision tasks and convolutional long short-term memory (ConvLSTM) has demonstrated its superb performance in such spatiotemporal problems. Nevertheless, the hierarchical data structures (e.g., human body parts and vessel/airway tree in biomedical images) in various tasks cannot be properly modeled by sequential models. Thus, ConvLSTM is not suitable for analyzing tree-structured image data that has a rich relation among its elements. In order to address this limitation, we present a tree-structured ConvLSTM model for tree-structured image analysis which can be trained end-to-end. To demonstrate its effectiveness, we first evaluate the proposed tree-structured ConvLSTM model on a synthetic Tree-Moving-MNIST dataset for tree-structured modeling. Experimental results demonstrate the superiority of the tree-structured ConvLSTM model for tree-structured image analysis compared with other alternatives. Additionally, we present a tree-structured segmentation framework which consists of a tree-structured ConvLSTM layer and an attention fully convolutional network (FCN) model. The proposed framework is validated on four large-scale coronary artery datasets. The results demonstrate the effectiveness and efficiency of the proposed method, showing its potential use cases in the analysis of tree-structured image data.

show abstract

“…The paper (Yunsi, Lahcen and Azzouz Mohamed, 2023) explores the application of Transformer neural networks to predict cryptocurrency prices. The paper (Du, Côté and Liu, 2023) proposes a novel method based on the self-attention mechanism containing of two diagonally-masked self-attention blocks that learn missing values from a weighted combination of temporal and feature dependencies.…”

Section: Researchers Have Proposed Adaptations Like the Temporal Fusi...mentioning

confidence: 99%

Tuning of Novel Fractional Order Fuzzy PID Controller for Automatic Voltage Regulator using Grasshopper Optimization Algorithm

Khaniki

Hadi

Manthouri

2021

mjee

View full text Add to dashboard Cite

One of the essential pieces of equipment in the power system is the Automatic Voltage Regulator (AVR) or synchronous generator excitation. The system's goal is to maintain the terminal voltage of the synchronous generator at the desired level. AVR is inherently uncertain. Hence, the proposed controller should be able to handle the problem. In this paper, Fractional Order Fuzzy PID (FOFPID) controller has been employed to control the system. To enhance the controller's performance, the Grasshopper Optimization Algorithm (GOA) is used to tune the controller's parameters. Unlike other methods, the FOFPID controller gains are not constant and alter in different operating conditions. The robustness of the controller has been investigated, and the comparative results show that the proposed controller has a better performance against other methods.

show abstract

SAITS: Self-attention-based imputation for time series

Cited by 109 publications

References 13 publications

ADF & TransApp: A Transformer-Based Framework for Appliance Detection Using Smart Meter Consumption Series

ADF & TransApp: A Transformer-Based Framework for Appliance Detection Using Smart Meter Consumption Series

Attention-driven tree-structured convolutional LSTM for high dimensional data understanding

Tuning of Novel Fractional Order Fuzzy PID Controller for Automatic Voltage Regulator using Grasshopper Optimization Algorithm

Contact Info

Product

Resources

About