MultiPub: Latency and Cost-Aware Global-Scale Cloud Publish/Subscribe

Gascon‐Samson, Julien; Kienzle, Jörg; Kemme, Bettina

doi:10.1109/icdcs.2017.203

Cited by 9 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Beyond the geo-context related work, there is also related work on (IoT) data distribution approaches and techniques; some of these also consider a subset of the above introduced geo-context dimensions. We see the most closely related approaches in the area of pub/sub, e.g., in IoT [26,27,28,29,30] but also for applications ranging from enterprise to web computing [31,32,33,34,35,36,37,38,39]. In addition, there is also non-pub/sub related work on data distribution in general, e.g., [40,41,42]; neither of these explicitly captures geo-context information.…”

Section: Rwmentioning

confidence: 99%

GeoBroker: Leveraging geo-contexts for IoT data distribution

Hasenburg

Bermbach

2020

Computer Communications

View full text Add to dashboard Cite

In the Internet of Things, the relevance of data often depends on the geographic context of data producers and consumers. Today's data distribution services, however, mostly focus on data content and not on geo-context, which could help to reduce the dissemination of excess data in many IoT scenarios. In this paper, we propose to use the geo-context information associated with devices to control data distribution. We define what geocontext dimensions exist and compare our definition with concepts from related work.Furthermore, we designed GeoBroker, a data distribution service that uses the location of things, as well as geofences for messages and subscriptions, to control data distribution. This way, we enable new IoT application scenarios while also increasing overall system efficiency for scenarios where geo-contexts matter by delivering only relevant messages. We evaluate our approach based on a proof-of-concept prototype and several experiments.

show abstract

Section: Rwmentioning

confidence: 99%

GeoBroker: Leveraging geo-contexts for IoT data distribution

Hasenburg

Bermbach

2020

Computer Communications

View full text Add to dashboard Cite

show abstract

“…Unlike this work which focuses on only a 3-node chain and focuses more on balancing the load on the brokers, our work focuses on the placement of competing, arbitrary DAGs comprising a workflow of stream processing operators on edge resources. MultiPub [25] is an effort to find the optimal placement of topics across geographically distributed datacenters for ensuring per-topic 90th percentile latency of data delivery. Although we are also interested in 90th percentile latencies, this work considers only inter-datacenter network latencies and does not consider edge networks.…”

Section: Related Workmentioning

confidence: 99%

Linearize, predict and place

Khare

Sun

Gascon‐Samson³

et al. 2019

Proceedings of the 4th ACM/IEEE Symposium on Edge Computing

Self Cite

View full text Add to dashboard Cite

Many IoT applications found in cyber-physical systems, such as smart grids, must take control actions in response to critical events, such as supply-demand mismatch, which requires low-latency processing of streaming data for rapid event detection and anomaly remediation. These streaming applications generally take the form of directed acyclic graphs (DAGs), where vertices represent operators and edges represent the flow of data between these operators. Edge computing has recently attracted significant attention as a means to readily meet the requirements of latency-critical IoT applications due to its ability to provide low-latency processing near the source of data. To accrue the benefits of edge computing, the constituent operators of these applications must be placed in a manner that intelligently trades-off inter-operator communication costs with the cost of interference incurred due to co-location of operators on the same resource-constrained edge devices. To address these challenges and to substantially simplify the placement problem for DAGs of arbitrary sizes and topologies, we present an algorithm that first transforms any arbitrary stream processing DAG into an approximate set of linear chains. Subsequently, a data-driven latency prediction model for co-located linear chains is used to inform the placement of operators such that the makespan, defined as the maximum latency of all paths in the DAG, is minimized. We empirically evaluate our algorithm using a variety of DAG placement scenarios on a Beagle Bone cluster, which is representative of an edge computing environment.

show abstract

“…Dynamoth [8], is a dynamic, scalable, channelbased pub/sub that proposes a software layer to load balance large number of publishers that publishes messages across multiple brokers which can be deployed as a cloud service. Multipub [9] is another study that attempts to scale dynamically depending on the current communication demands with the multiple-brokers deployed in the cloud. MultiPub proposes a flexible pub/sub system for latency-constrained, globally distributed applications.…”

Section: Related Workmentioning

confidence: 99%

“…With the growth of distributed applications that use the pub/sub model, methods to provision resources in the pub/sub system in a flexible manner has gained interest. This is particularly the case when these systems are deployed as a cloud service [9], [13], [8]. In [13] the problem of resource allocation in a scalable pub/sub system is formulated as an optimization problem with different objective functions.…”

Section: Introductionmentioning

confidence: 99%

Auto-Tuned Publisher in a Pub/Sub System: Design and Performance Evaluation

Balasubramanian¹,

Ghosal

Sharath³

et al. 2018

2018 IEEE International Conference on Autonomic Computing (ICAC)

View full text Add to dashboard Cite

Pub/sub systems form the underlying framework for many distributed applications. It consists of one or more publishers that publish to a broker from which subscriber can retrieve the published content. Many large and small distributed applications including social networking applications use the pub/sub model. In this paper we consider a pub/sub system in which the publisher, the broker, and the subscriber are in different administrative domains. While general pub/sub systems provide reliability of message delivery, good end-to-end latency performance in a multi-domain environment require that the pub/sub system adapt to workload changes and bottlenecks in the different sub-systems. This study is motivated by two applications. First, a pub/sub based Simple Lookup Service (sLS) that is used in perfSONAR to provide information about network performance in R&E networks. Second, the pub/sub system that is used to distribute alerts generated in the data pipeline in the Zwicky Transient Factory (ZTF). We consider a publisher with a multi-threaded architecture that uses batching to coalesce messages over some variable polling period. We propose a control algorithm that auto-tunes the batch-processing parameters namely, the batch size and the polling interval, to the input message load and to any upstream broker-side congestion that minimizes end-to-end latency and maximizes throughput. Using a detailed simulation model, we demonstrate the performance of the control algorithm for different scenarios. We then study the performance using a real-trace obtained from the Simple Lookup Service (sLS). We show that the proposed algorithm quickly adapts to rapid changes in the workload and yields lower mean end-to-end delay performance when compared with the delays in the current deployment.

show abstract

MultiPub: Latency and Cost-Aware Global-Scale Cloud Publish/Subscribe

Cited by 9 publications

References 14 publications

GeoBroker: Leveraging geo-contexts for IoT data distribution

GeoBroker: Leveraging geo-contexts for IoT data distribution

Linearize, predict and place

Auto-Tuned Publisher in a Pub/Sub System: Design and Performance Evaluation

Contact Info

Product

Resources

About