Autonomous Unknown-Application Filtering and Labeling for DL-based Traffic Classifier Update

Zhang, Jielun; Li, Fuhao; Feng, Yi; Wu, Hongyu

doi:10.1109/infocom41043.2020.9155292

Cited by 73 publications

(26 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In terms of (i) accuracy, for the 1-dimensional time-series problem of the traffic classification use-case, OSR techniques are well dispersed: gradient-based [13] performs the best (significantly better than SoftMax [10] and OpenMax [11], with very similar complexity) and input-clustering [5] the worst (due to the "curse of dimensionality" and the high number of clusters, over which output-clustering [12] only provides a very limited advantage). In the bi-dimensional image use-case, performance of top-methods are in a closer range, but the gradient-based method still consistently shows superior performance.…”

Section: B Experimental Resultsmentioning

confidence: 99%

“…To counter this problem, OpenMax [11] revises SoftMax activation vectors adding a special "synthetic" unknown class (with weigths induced by Weibull modeling). To overcome curse of dimensionality, [12] performs clustering at the output of the neural network, with a PCA dimensionality reduction. In [13] we combine XAI and OSR by using a gradient-based method, where gradient backpropagation (limited to the last DL layer to keep computation simple) is used to detect and explain large changes in the feed-forward model due that are due to the new input.…”

Section: B Quality Assessment State-of-the-artmentioning

confidence: 99%

“…We thus additionally consider an image-related use-case, which (i) is performed on public datasets thus reproducible; additionally, (ii) image recognition is a network application, when edge devices stream images to the cloud for processing, where novelty discovery is a tool to assess the fitness of the model to environment; finally, (iii) the use of image recognition allows to appreciate generality of the methods. In particular, the MNIST handwritten digits and characters dataset is consistently used in the OSR literature [10]- [12] as a benchmark. In this case, the problem is for any input, to assess whether is part of the handwritten digits and characters shown to the model during training, or if is an entirely new symbol.…”

Section: Example Aiops Use-casesmentioning

confidence: 99%

See 2 more Smart Citations

Quality Monitoring and Assessment of Deployed Deep Learning Models for Network AIOps

Yang¹,

Rossi²

2022

Preprint

View full text Add to dashboard Cite

Artificial Intelligence (AI) has recently attracted a lot of attention, transitioning from research labs to a wide range of successful deployments in many fields, which is particularly true for Deep Learning (DL) techniques. Ultimately, DL models being software artifacts, they need to be regularly maintained and updated: AIOps is the logical extension of the DevOps software development practices to AI-software applied to network operation and management. In the lifecycle of a DL model deployment, it is important to assess the quality of deployed models, to detect "stale" models and prioritize their update. In this article, we cover the issue in the context of network management, proposing simple yet effective techniques for (i) quality assessment of individual inference, and for (ii) overall model quality tracking over multiple inferences, that we apply to two use cases, representative of the network management and image recognition fields.

show abstract

Section: B Experimental Resultsmentioning

confidence: 99%

Section: B Quality Assessment State-of-the-artmentioning

confidence: 99%

Section: Example Aiops Use-casesmentioning

confidence: 99%

See 1 more Smart Citation

Quality Monitoring and Assessment of Deployed Deep Learning Models for Network AIOps

Yang¹,

Rossi²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…OpenMax [28] revises SoftMax activation vectors adding a special "synthetic" unknown class (by using weigthing induced by Weibull modeling). Alternative approaches include the use of Extreme Value Machine (EVM) [27], based of Extreme Value Theory (EVT), and, more recently, clustering on the CNN feature vectors with a PCA reduction of dimension [30].…”

Section: ) Outputmentioning

confidence: 99%

“…where, as previously, d(•) and F V are a distance metric and an arbitrary threshold respectively. This technique is standard in the DL domain, and can be considered as a state of the art for DL-based clustering [30].…”

Section: ) Input Based Clustering (Ml and Dl)mentioning

confidence: 99%

Deep Learning and Traffic Classification: Lessons learned from a commercial-grade dataset with hundreds of encrypted and zero-day applications

Yang,

Finamore,

Jun

et al. 2021

Preprint

View full text Add to dashboard Cite

The increasing success of Machine Learning (ML) and Deep Learning (DL) has recently re-sparked interest towards traffic classification. While classification of known traffic is a well investigated subject with supervised classification tools (such as ML and DL models) are known to provide satisfactory performance, detection of unknown (or zero-day) traffic is more challenging and typically handled by unsupervised techniques (such as clustering algorithms).In this paper, we share our experience on a commercialgrade DL traffic classification engine that is able to (i) identify known applications from encrypted traffic, as well as (ii) handle unknown zero-day applications. In particular, our contribution for (i) is to perform a thorough assessment of state of the art traffic classifiers in commercial-grade settings comprising few thousands of very fine grained application labels, as opposite to the few tens of classes generally targeted in academic evaluations. Additionally, we contribute to the problem of (ii) detection of zero-day applications by proposing a novel technique, tailored for DL models, that is significantly more accurate and light-weight than the state of the art.Summarizing our main findings, we gather that (i) while ML and DL models are both equally able to provide satisfactory solution for classification of known traffic, however (ii) the nonlinear feature extraction process of the DL backbone provides sizeable advantages for the detection of unknown classes.Index Terms-Machine learning, Data mining and (big) data analysis, Network monitoring and measurements.

show abstract

Mobile Encrypted Traffic Classification Based on Message Type Inference

Chen

Zang

Zhang

et al. 2021

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

View full text Add to dashboard Cite

Autonomous Unknown-Application Filtering and Labeling for DL-based Traffic Classifier Update

Cited by 73 publications

References 27 publications

Quality Monitoring and Assessment of Deployed Deep Learning Models for Network AIOps

Quality Monitoring and Assessment of Deployed Deep Learning Models for Network AIOps

Deep Learning and Traffic Classification: Lessons learned from a commercial-grade dataset with hundreds of encrypted and zero-day applications

Mobile Encrypted Traffic Classification Based on Message Type Inference

Contact Info

Product

Resources

About