Neural Tensor Completion for Accurate Network Monitoring

Xie, Kun; Lu, Huali; Wang, Xin; Xie, Gaogang; Ding, Yong; Xie, Dong; Wen, Junhai; Zhang, Dafang

doi:10.1109/infocom41043.2020.9155366

Cited by 21 publications

(11 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• NTC [1]: the network traffic recovery model combines deep learning model (3D-CNN) and CP decompositionbased approach.…”

Section: A Datasets and Baseline Methodsmentioning

confidence: 99%

“…IV. DOWNSTREAM APPLICATIONS Although there are many proposed methods for imputing or recovering missing values in the network data [1], [4], most of them concentrated on improving the imputation error without considering the performance of downstream applications. In this section, we present how Traffic Engineering leverages network traffic imputation.…”

Section: Spatial Feature Learning With Gcnmentioning

confidence: 99%

“…Building an effective machine learning model requires high accuracy in the historical measurement data. However, missing values in networking data are inevitable due to unexpected accidents or intended purposes (e.g., sparse monitoring [1]). This inappropriate data imposes a significant impact on the performance of downstream applications.…”

Section: Introductionmentioning

confidence: 99%

“…However, E 2 GAN only uses a unidirectional recurrent model and does not consider the correlation between the time series. Most recently, motivated by CP decomposition, NTC model was introduced in [1], which was specifically designed for imputing network traffic. However, by taking the index of each observed data as the input, NTC suffers from a scalability issue when the number of observed data in the training set is significantly large.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

GCRINT: Network Traffic Imputation Using Graph Convolutional Recurrent Neural Network

Nguyen

et al. 2021

ICC 2021 - IEEE International Conference on Communications

View full text Add to dashboard Cite

Missing values appear in most multivariate time series, especially in the monitored network traffic data due to high measurement cost and unavoidable loss. In the networking fields, missing data prevents advanced analysis and downgrades downstream applications such as traffic engineering and anomaly detection. Despite the great potential, existing imputation approaches based on tensor decomposition and deep learning techniques have shown limitations in addressing missing values of traffic data due to its dynamic behavior. In this paper, we propose Graph Convolutional Recurrent Neural Network for Imputing Network Traffic (GCRINT), a combination between Recurrent Neural Network (RNN) and Graph Convolutional Neural Network, for filling the missing values of network traffic data. We use a bidirectional Long Short-Term Memory network and Graph Neural Network to efficiently learn the spatial-temporal correlations in partially observed data. We conducted extensive experiments to evaluate our model by using two different datasets and various missing scenarios. The experiment results show that GCRINT achieves significantly low imputation errors and reduces the error by 35% compared to the state-of-the-art methods. GCRINT also helps to obtain a stable performance in the traffic engineering problem.

show abstract

“…• NTC [1]: the network traffic recovery model combines deep learning model (3D-CNN) and CP decompositionbased approach.…”

Section: A Datasets and Baseline Methodsmentioning

confidence: 99%

Section: Spatial Feature Learning With Gcnmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

GCRINT: Network Traffic Imputation Using Graph Convolutional Recurrent Neural Network

Nguyen

et al. 2021

ICC 2021 - IEEE International Conference on Communications

View full text Add to dashboard Cite

show abstract

“…Neural network-based tensor completion methods [2,21,36,37] have been proposed to enhance the estimation accuracy. They have strong generalization capability [24], and capture non-linearity hidden in a tensor.…”

Section: Introductionmentioning

confidence: 99%

Influence-guided Data Augmentation for Neural Tensor Completion

Kim

Rossi

et al. 2021

Proceedings of the 30th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

How can we predict missing values in multi-dimensional data (or tensors) more accurately? The task of tensor completion is crucial in many applications such as personalized recommendation, image and video restoration, and link prediction in social networks. Many tensor factorization and neural network-based tensor completion algorithms have been developed to predict missing entries in partially observed tensors. However, they can produce inaccurate estimations as real-world tensors are very sparse, and these methods tend to overfit on the small amount of data. Here, we overcome these shortcomings by presenting a data augmentation technique for tensors. In this paper, we propose DAIN, a general data augmentation framework that enhances the prediction accuracy of neural tensor completion methods. Specifically, DAIN first trains a neural model and finds tensor cell importances with influence functions. After that, DAIN aggregates the cell importance to calculate the importance of each entity (i.e., an index of a dimension). Finally, DAIN augments the tensor by weighted sampling of entity importances and a value predictor. Extensive experimental results show that DAIN outperforms all data augmentation baselines in terms of enhancing imputation accuracy of neural tensor completion on four diverse real-world tensors. Ablation studies of DAIN substantiate the effectiveness of each component of DAIN. Furthermore, we show that DAIN scales near linearly to large datasets. CCS CONCEPTS• Computing methodologies → Factorization methods; Neural networks; • Information systems → Data mining.

show abstract