Another Outlier Bites the Dust: Computing Meaningful Aggregates in Sensor Networks

Abstract

Abstract-Recent work has demonstrated that readings provided by commodity sensor nodes are often of poor quality. In order to provide a valuable sensory infrastructure for monitoring applications, we first need to devise techniques that can withstand "dirty" and unreliable data during query processing. In this paper we present a novel aggregation framework that detects suspicious measurements by outlier nodes and refrains from incorporating such measurements in the computed aggregate values. We consider different definitions of an outlier node, based on the notion of a user-specified minimum support, and discuss techniques for properly routing messages in the network in order to reduce the bandwidth consumption and the energy drain during the query evaluation. In our experiments using real and synthetic traces we demonstrate that: (i) a straightforward evaluation of a user aggregate query leads to practically meaningless results due to the existence of outliers; (ii) our techniques can detect and eliminate spurious readings without any application specific knowledge of what constitutes normal behavior; (iii) the identification of outliers, when performed inside the network, significantly reduces bandwidth and energy drain compared to alternative methods that centrally collect and analyze all sensory data; and (iv) we can significantly reduce the cost of the aggregation process by utilizing simple statistics on outlier nodes and reorganizing accordingly the collection tree.

SeTraStream: Semantic-Aware Trajectory Construction over Streaming Movement Data

“…We already noted the particular utility of the correlation coefficient on the discovery of trends [5][10][11] [19], and thus (in our context) patterns in the movement data. In this processing phase movement feature vectors composing each workpiece essentially form a pair of matrices for which correlation computation needs to be conducted.…”

“…The choice of scc is motivated by the fact that its stem, corr, possesses the ability to indicate the similarity of the trends that are profound in the examined vectors rather than relying on their absolute values [5][10][11] [19]. Hence, it provides an appropriate way to identify (dis)similar patterns in the complementary vectors and can be generalized in order to detect similar patterns between movement feature vectors in their entirety.…”

“…In the context of streaming data, online filtering & smoothing of streaming tuples has become a hot topic [5][10] [11][15] [19]. Different from the focus of prior works on data accuracy and distribution estimation, our primary concern of cleaning streaming movement data is refining the data points that have substantial distortion of movement features for computing semantic trajectories 1 .…”

Subjective Logic-Based Anomaly Detection Framework in Wireless Sensor Networks

“…It returns the aggregate results excluding anomaly, which is also maintained and sent to the users. Furthermore, authors in [9] define minimum support MinSupp, which is the minimum count of sensor nodes to prove the data of the node being normal or anomalous. For some node holds on anomalous data, if it has MinSupp number of nodes whose data are similar to it, it is determined that some events happen, otherwise it is faulty data.…”

“…In this process, existing solutions, including voting algorithms [6,7] and aggregation frameworks [8][9][10] which detect anomaly in the process of aggregating data, provide neighbors' opinions being just normal and anomalous. However, no neighbor can always say that the data of the node are absolutely normal or anomalous, and something is neglected by previous works which we call uncertainty.…”