Structured nonnegative matrix factorization for traffic flow estimation of large cloud networks

Atif, Syed Muhammad; Gillis, Nicolas; Qazi, Sameer; Naseem, Imran

doi:10.1016/j.comnet.2021.108564

Cited by 3 publications

(4 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Framework for predicting data with AR. • ARNMF [4]: Non-negative constrained version of TRMF. Our purpose in using this method as a baseline is to verify whether NMF is the main reason for the better performance achieved by RTNMFFM.…”

Section: Competitorsmentioning

confidence: 99%

See 1 more Smart Citation

Robust Multi-Dimensional Time Series Forecasting

Shen,

He,

Qin

2024

Entropy

View full text Add to dashboard Cite

Large-scale and high-dimensional time series data are widely generated in modern applications such as intelligent transportation and environmental monitoring. However, such data contains much noise, outliers, and missing values due to interference during measurement or transmission. Directly forecasting such types of data (i.e., anomalous data) can be extremely challenging. The traditional method to deal with anomalies is to cut out the time series with anomalous value entries or replace the data. Both methods may lose important knowledge from the original data. In this paper, we propose a multidimensional time series forecasting framework that can better handle anomalous values: the robust temporal nonnegative matrix factorization forecasting model (RTNMFFM) for multi-dimensional time series. RTNMFFM integrates the autoregressive regularizer into nonnegative matrix factorization (NMF) with the application of the L2,1 norm in NMF. This approach improves robustness and alleviates overfitting compared to standard methods. In addition, to improve the accuracy of model forecasts on severely missing data, we propose a periodic smoothing penalty that keeps the sparse time slices as close as possible to the time slice with high confidence. Finally, we train the model using the alternating gradient descent algorithm. Numerous experiments demonstrate that RTNMFFM provides better robustness and better prediction accuracy.

show abstract

Section: Competitorsmentioning

confidence: 99%

“…For example, the speed of vehicle movement and customer electricity consumption are usually stored in a nonnegative matrix. In addition to IoT data, multidimensional time series data are similarly generated in some fields, such as e-commerce [ 3 ], web traffic [ 4 ], and the biomedical field [ 5 ].…”

Section: Introductionmentioning

confidence: 99%

Robust Multi-Dimensional Time Series Forecasting

Shen,

He,

Qin

2024

Entropy

View full text Add to dashboard Cite

show abstract

“…Within this context, various neural network models have been proposed for solving the TME problem, including a state-space recurrent multilayer perceptron (RMLP) [30], a back-propagation neural network (BPNN) combined with the iterative proportional fitting procedure (IPFP) [31] or an auto-regressive (AR) model [32], a non-linear auto-regressive exogenous model (NARX) together with the genetic algorithm (GA) [33], and a deep belief network (DBN) [34]. More recent works have attempted to incorporate routing information and topological network structure into the neural network input, such as the Moore-Penrose inverse of the routing matrix multiplied with link load vector [35] and graph-embedding [36]- [38]. The latter has been combined with convolutional neural networks [36], [37] and nonnegative matrix factorization (NMF) [38].…”

Section: Related Workmentioning

confidence: 99%

“…More recent works have attempted to incorporate routing information and topological network structure into the neural network input, such as the Moore-Penrose inverse of the routing matrix multiplied with link load vector [35] and graph-embedding [36]- [38]. The latter has been combined with convolutional neural networks [36], [37] and nonnegative matrix factorization (NMF) [38]. A feedforward back-propagation neural network trained with the Levernberg-Marquardt algorithm has also been proposed [39].…”

Section: Related Workmentioning

confidence: 99%

Generative Deep Learning Techniques for Traffic Matrix Estimation From Link Load Measurements

Kakkavas,

Fryganiotis,

Karyotis

et al. 2024

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

Traffic matrices (TMs) contain crucial information for managing networks, optimizing traffic flow, and detecting anomalies. However, directly measuring traffic to construct a TM is resource-intensive and computationally expensive. A more practical approach involves estimating the TM from readily available link load measurements, which falls under the category of inferential network monitoring based on indirect measurements known as network tomography. This paper focuses on solving the problem of estimating the traffic matrix from link loads by utilizing deep generative models. The proposed models are trained using historical data-specifically, previously observed TMs-and are then leveraged to transform traffic matrix estimation (TME) into a simpler minimization problem in a lower-dimensional latent space. This transformed problem can be efficiently solved using a gradient-based optimizer. Our work aims to examine and test different model architectures and optimization approaches. The performance of the proposed methods is comparatively evaluated over a comprehensive set of suitable metrics on two publicly available datasets comprising actual traffic matrices obtained from real backbone networks. In addition, we compare our approach with a state-of-the-art method previously published in the literature.

show abstract