No abstract

Abstract. Subscription adaptations are becoming increasingly important across many content-based publish/subscribe (CPS) applications. In algorithmic high frequency trading, for instance, stock price thresholds that are of interest to a trader change rapidly, and gains directly hinge on the reaction time to relevant fluctuations. The common solution to adapt a subscription consists of a re-subscription, where a new subscription is issued and the superseded one canceled. This is ineffective, leading to missed or duplicate events during the transition. In this paper, we introduce the concept of parametric subscriptions to support subscription adaptations. We propose novel algorithms for updating routing mechanisms effectively and efficiently in classic CPS broker overlay networks. Compared to re-subscriptions, our algorithms significantly improve the reaction time to subscription updates and can sustain higher throughput in the presence of high update rates. We convey our claims through implementations of our algorithms in two CPS systems, and by evaluating them on two different real-world applications.

Content-based publish/subscribe (CPS) is an appealing abstraction for building scalable distributed systems, e.g., message boards, intrusion detectors, or algorithmic stock trading platforms. More recently, extensions of the abstraction have been proposed for location-based services like vehicular networks, mobile social networking, etc.Although current CPS middleware systems are dynamic in the way they support the joining and leaving of publishers and subscribers, they fall short in supporting subscription adaptations. These are becoming increasingly important across many CPS applications. In algorithmic high frequency trading, for instance, stock price thresholds that are of interest to a trader change rapidly, and gains directly hinge on the reaction time to relevant fluctuations rather than fixed values. In location-aware applications, a subscription is a function of the subscriber location (e.g. GPS coordinates) which inherently changes during motion.The common solution to adapt a subscription consists in a re-subscription, where a new subscription is issued and the superseded one canceled. This incurs substantial overhead in CPS middleware systems, and leads to missed or duplicate events during the transition. In this paper, we explore the concept of parametric subscriptions for capturing subscription adaptations. We discuss desirable and feasible guarantees for corresponding support, and propose novel algorithms for updating routing mechanisms effectively and efficiently in classic decentralized CPS broker overlay networks. Compared to re-subscriptions, our algorithms significantly improve the reaction time to subscription updates without hampering throughput or latency under high update rates. We investigate pathological cases of high frequency subscription oscillations, which could significantly decrease the throughput of CPS systems thereby affecting other subscribers. We propose and evaluate approximations techniques to detect such oscillations and mitigate the event filtering burden they place on the CPS system.We convey analyze the benefits of our support through implementations of our algorithms in two CPS systems, and by evaluating our algorithms on two different application scenarios.

Aggregation of computed sets of results fundamentally underlies the distillation of information in many of today's big data applications. To this end there are many systems which have been introduced which allow users to obtain aggregate results by aggregating along communication structures such as trees, but they do not focus on optimizing performance by optimizing the underlying structure to perform the aggregation.We consider two cases of the problem -aggregation of (1) single blocks of data, and of (2) streaming input. For each case we determine which metric of "fast" completion is the most relevant and mathematically model resulting systems based on aggregation trees to optimize that metric. Our assumptions and model are laid out in depth. From our model we determine how to create a provably ideal aggregation tree (i.e., with optimal fanin) using only limited information about the aggregation function being applied. Experiments in the Amazon Elastic Compute Cloud (EC2) confirm the validatity of our models in practice.

No abstract