Semantic feature selection for network telemetry event description

Abstract

Model driven telemetry (MDT) enables the real-time collection of hundreds of thousands of counters on large-scale networks, with contextual information to each counter provided in the telemetry data structure definition. Explaining network events in such datasets implies substantial analysis by a domain expert. This paper presents an semantic feature selection method, to find the most important counters which describe a given event in a telemetry dataset, and facilitate the explanation process. This paper proposes a metric for estimating the importance of features in a dataset with descriptive feature names, to find those that are most meaningful to a human. With this estimation, this paper presents a cross-entropy based metric describing the quality of a selection of counters, which is combined with the data behavior to define an optimization goal. The computation of optimal selections distills intelligible and precise selections of counters with adjustable verbosity, and describes events with a few selected counters outlining the root cause of network events.

Understanding Semantics in Feature Selection for Fault Diagnosis in Network Telemetry Data

“…Although the process of selecting original features for fault diagnosis is similar, the purpose of explainable AI methods is to describe the reason for a given classification (that is, for their own decisions), whereas the objective of this study is to describe the underlying data itself. A first specification and preliminary results of this method for fault diagnosis was presented in [18].…”

“…This paper presents an evaluation of the semantic feature selection method presented in [18] on network telemetry datasets, a hybrid selection process which combines datadriven metrics and semantic analysis of meta-data. This approach produces a representation for network fault events, extracted from the telemetry available, that can be used for fault diagnosis.…”

“…This section presents a generalized approach to the semantic importance estimation presented in [18] which uses meta-data information contained in feature names.…”

“…Section III having illustrated that such relevance cannot be retrieved exclusively from data, this paper suggests to estimate the relative importance of features for fault diagnosis by exploiting available meta-data, which carries semantic information. The applied methodology extends the feature name based semantic importance estimation of [18].…”

“…The semantic feature selection is defined as an optimization process which jointly maximizes the data-driven change metric and the cross entropy based semantic importance estimation, as shown in Figure 3. The final optimization objective is defined as [18]:…”