A Novel Approach Based on Matrix Factorization for Recovering Missing Time Series Sensor Data

Song, Xiaoxiang; Guo, Yan; Li, Ning; Yang, Sixing

doi:10.1109/jsen.2020.3004186

Cited by 12 publications

(6 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MF models such as non-negative matrix factorization (NMF) [ 22 ] and more general forms of tensor factorization (CP, Tucker) have been widely used for complex time-stamped events [ 9 ]. On time series data, matrix factorization is widely used for dimensionality reduction [ 23 ] and data imputation [ 24 ].…”

Section: Related Workmentioning

confidence: 99%

Robust Multi-Dimensional Time Series Forecasting

Shen,

He,

Qin

2024

Entropy

View full text Add to dashboard Cite

Large-scale and high-dimensional time series data are widely generated in modern applications such as intelligent transportation and environmental monitoring. However, such data contains much noise, outliers, and missing values due to interference during measurement or transmission. Directly forecasting such types of data (i.e., anomalous data) can be extremely challenging. The traditional method to deal with anomalies is to cut out the time series with anomalous value entries or replace the data. Both methods may lose important knowledge from the original data. In this paper, we propose a multidimensional time series forecasting framework that can better handle anomalous values: the robust temporal nonnegative matrix factorization forecasting model (RTNMFFM) for multi-dimensional time series. RTNMFFM integrates the autoregressive regularizer into nonnegative matrix factorization (NMF) with the application of the L2,1 norm in NMF. This approach improves robustness and alleviates overfitting compared to standard methods. In addition, to improve the accuracy of model forecasts on severely missing data, we propose a periodic smoothing penalty that keeps the sparse time slices as close as possible to the time slice with high confidence. Finally, we train the model using the alternating gradient descent algorithm. Numerous experiments demonstrate that RTNMFFM provides better robustness and better prediction accuracy.

show abstract

Section: Related Workmentioning

confidence: 99%

Robust Multi-Dimensional Time Series Forecasting

Shen,

He,

Qin

2024

Entropy

View full text Add to dashboard Cite

show abstract

“…Type Dataset Purpose Method [14] External events Real data Reconstructing missing data Algorithm [12] External events Real data and noise Reconstructing missing data Algorithm [15] External events Simulated data Reconstructing missing data Algorithm [16] External events Real data Reconstructing missing data Machine learning [13] External events Real data and noise Reconstructing lost data Deep learning [17] Internal events Real data Reconstructing data Algorithm [18] Internal events Real data Reconstructing data Bidirectional recurrent neural network [19] Internal events Real data Reconstructing data Algorithm [20] Internal events Real data and noise Denoising data Algorithm [21] Internal events Real data and noise Reconstructing data Convolutional neural network…”

Section: Paper Yearmentioning

confidence: 99%

Noise-Tolerant Data Reconstruction Based on Convolutional Autoencoder for Wireless Sensor Network

Lai,

Tran,

Cho

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Maintaining data dependability within wireless sensor network (WSN) systems has significant importance. Nevertheless, the deployment of systems in unattended and hostile areas poses a major challenge in dealing with noise. Consequently, several investigations have been conducted to address the issue of noise-affected data recovery. Nevertheless, previous research has primarily focused on the internal noise of the system. Neglecting to include external factors that impact the WSN system in the study might lead to findings that are not true to reality. Hence, this research takes into account both internal and external noise factors, such as rain, fog, or snow conditions. Moreover, in order to maintain the temporal characteristics and intersensor relationships, the data from multiple sensor nodes are consolidated into a two-dimensional matrix format. The stacked convolutional autoencoder (SCAE) model is proposed, which has the capability to extract data features. The stack design of the SCAE enables it to effectively mitigate the issue of vanishing gradients. Moreover, the weight sharing approach used between the two subnetworks also enhances the efficiency of the weight initialization procedure. Thorough experiments are conducted using both simulated WSN systems and real-world sensing data. Experimental results demonstrate that the SCAE outperforms existing methods for reconstructing noisy data.

show abstract

“…Initially, data are encapsulated into a matrix then the missing data can be identified through PCI and the DCT is used for data recovery. To recover the missing time series data MF-EALS [ 17 ], matrix minimization, ST correlation with low-rank matrices [ 33 ], and matrix completion [ 34 ] methods are implemented in sensor networks. Depending on the data patterns and correlation among the time series data the missing data have been recovered.…”

Section: Related Workmentioning

confidence: 99%

“…The proposed approach has been simulated in real-world sensory data sets, which resulted in more than 3% of reliability as compared with other data recovery approaches. Those are bidirectional long short-term memory (BI-LSTM) [ 16 ], matrix factorization alternating least squares (MF-EALS) [ 17 ], and data reconstruction using temporal stability guided matrix completion (DRTSMC) [ 18 ] recovery approaches. The existing approaches utilized redundancy and periodicity to predict the specific node data depending on the past data, resulting in low prediction stability and biased predictions.…”

Section: Introductionmentioning

confidence: 99%

A Deep Learning Based Data Recovery Approach for Missing and Erroneous Data of IoT Nodes

Vedavalli

2022

Sensors

View full text Add to dashboard Cite

Internet of things (IoT) nodes are deployed in large-scale automated monitoring applications to capture the massive amount of data from various locations in a time-series manner. The captured data are affected due to several factors such as device malfunctioning, unstable communication, environmental factors, synchronization problem, and unreliable nodes, which results in data inconsistency. Data recovery approaches are one of the best solutions to reduce data inconsistency. This research provides a missing data recovery approach based on spatial-temporal (ST) correlation between the IoT nodes in the network. The proposed approach has a clustering phase (CL) and a data recovery (DR) phase. In the CL phase, the nodes can be clustered based on their spatial and temporal relationship, and common neighbors are extracted. In the DR phase, missing data can be recovered with the help of neighbor nodes using the ST-hierarchical long short-term memory (ST-HLSTM) algorithm. The proposed algorithm has been verified on real-world IoT-based hydraulic test rig data sets which are gathered from things speak real-time cloud platform. The algorithm shows approximately 98.5% reliability as compared with the other existing algorithms due to its spatial-temporal features based on deep neural network architecture.

show abstract

A Novel Approach Based on Matrix Factorization for Recovering Missing Time Series Sensor Data

Cited by 12 publications

References 25 publications

Robust Multi-Dimensional Time Series Forecasting

Robust Multi-Dimensional Time Series Forecasting

Noise-Tolerant Data Reconstruction Based on Convolutional Autoencoder for Wireless Sensor Network

A Deep Learning Based Data Recovery Approach for Missing and Erroneous Data of IoT Nodes

Contact Info

Product

Resources

About