A Communication-Efficient Distributed Data Structure for Top-k and k-Select Queries

Abstract

We consider the scenario of n sensor nodes observing streams of data. The nodes are connected to a central server whose task it is to compute some function over all data items observed by the nodes. In our case, there exists a total order on the data items observed by the nodes. Our goal is to compute the k currently lowest observed values or a value with rank in [(1 − ε)k, (1 + ε)k] with probability (1 − δ). We propose solutions for these problems in an extension of the distributed monitoring model where the server can send broadcast messages to all nodes for unit cost. We want to minimize communication over multiple time steps where there are m updates to a node's value in between queries. The result is composed of two main parts, which each may be of independent interest: * This work was partially supported by the German Research Foundation (DFG) within the Priority Program "Algorithms for Big Data" (SPP 1736).Consider a distributed sensor network which is a system consisting of a huge amount of nodes. Each node continuously observes its environment and measures information (e.g. temperature, pollution or similar parameters). We are interested in aggregations describing the current observations at a central server.To keep the server's information up to date, the server and the nodes can communicate with each other. In sensor networks, however, the amount of such communication is particularly crucial, as communication has the largest impact to energy consumption, which is limited due to battery capacities [11]. Therefore, algorithms aim at minimizing the (total) communication required for computing the respective aggregation function at the server.We consider several ideas to potentially lower the communication used. Each single computation of an aggregate should use as little communication as possible. Computations of the same aggregate should reuse parts of previous computations. Only compute aggregates, if necessary. Recall that the continuous monitoring model creates a new output as often as possible.

A Dynamic Distributed Data Structure for Top-k and k-Select Queries

“…This represents the case (with respect to the choice of φ) that a small number of messages and a large number of communication rounds are used. This observation is complemented by a lower bound of Ω(log n) in[4].…”

“…The distributed monitoring model introduced by Cormode, Muthukrishnan, and Yi in [8] allows exchanging single cast messages. The extension we use is the Distributed Monitoring Model with a Broadcast Channel (DMBC-Model) (proposed in [7] and exploited in [3,4,9,10]) which allows, in addition, to broadcast messages from the server to all nodes. Both types of communication are instantaneous and have unit cost per message.…”

“…In this paper we improve both the number of messages and communication rounds to k + log n + 2 and O(k + log N ) respectively. The techniques used in this paper are fundamentally based on [4] applying the idea of an inorder treewalk in a distributed searchtree and analyzing using a mixed distribution.…”