Rejoinder to “Statistical methods for network surveillance”

Abstract

their thoughtful comments. As we can all agree, the field of network surveillance is burgeoning, and there are many interesting research questions and methodological advances that are yet to be addressed. As such, we hope that this paper and these discussions lead to a meaningful dialogue and are an impetus for further work in the field of network surveillance.Dr Sengupta and Dr Woodall emphasize the importance of distinguishing between deterministic and stochastic networks. We completely agree; this is a very helpful way of classifying problems and organizing solution approaches. In the case of dynamic social networks, it can be important to model observations by "double-stochastic" schemes. To account for the interdependent nature of dynamics, hierarchical coupled hidden Markov models (HMMs) are suitable. 1,2 These models can be viewed as parallel rolled-out HMM chains coupled through cross-time and cross-chain conditional probabilities. The state of each variable depends on all of its parents in the previous time slice. Therefore, the processes assert temporal influence on each other's states; the interaction between the chains is informed by the internal states of the nodes, while the states of those nodes themselves are influenced by those interactions. This provides a feedback mechanism between individual and collective dynamics, which translates into a very rich behavior of the model.Dr Sengupta and Dr Woodall also ask How quickly can one react to an anomaly and what action can be taken? The detection delay depends on three factors: (i) the required false alarm rate (FAR), (ii) the intensity of the anomaly in traffic, and (iii) the level of data aggregation. As we discussed in Section 3, in order to reduce the detection delay, one has to lower thresholds in the anomaly-based intrusion detection system (IDS). However, this will lead to an increase of FAR and filtering of false detection based on the spectral signature algorithms becomes a critical aspect, which is the goal of the proposed anomaly signature IDS. Similarly, the threshold for the repeated joint likelihood ratio test discussed in Section 4 is computed based on a FAR target, illustrating again the inescapable trade-off between detection delay and FAR.It has been pointed out that the frequency with which observations are sampled deterministically influences the speed with which anomalies are identified. 3,4 A practical reality is that the sampling frequency is sometimes fixed by the incumbent data acquisition scheme. Such was the case regarding the five-minute sampling intervals for our Figure 2.The question of What action can be taken after detection? depends on a particular application. Typically, computer analysts should make a final decision. The plot in Figure 4 simply shows how the CUSUM statistic behaves for this particular attack. The goal is just to detect the first threshold exceedance, and then forward this information to analysts for making a final decision. If one needs to detect the end of the attack, then a different "anomaly disap...