BORDER: A Benchmarking Framework for Distributed MQTT Brokers

Abstract

MQTT, one of the most popular application layer protocols for the IoT, works according to a publish/subscribe paradigm where clients connect to a centralized broker. Sometimes (e.g., in high scalability and low latency applications), it is required to depart from such a centralized approach and move to a distributed one, where multiple MQTT brokers cooperate together. Many MQTT brokers (both open-source or commercially available) allow to create such a distributed environment: however, it is challenging to select the right solution due to the many available choices. This paper proposes therefore BORDER (Benchmarking framewORk for DistributEd MQTT bRokers), a framework for creating and evaluating distributed architectures of MQTT brokers with realistic and customizable network topologies. Based on isolated Docker containers and emulated network components, the framework provides quantitative metrics about the overall system performance, such as end-toend latency as well as network and physical resources consumed. We use BORDER to compare five of the most popular MQTT brokers that allow the creation of distributed architectures and we release it as an open-source project to allow for reproducible researches.

Design and implementation of an advanced MQTT broker for distributed pub/sub scenarios

“…For the test scenario, we deployed five brokers in the system in a fullmeshed physical topology. The network is created using the BORDER framework [33], an extension of Mininet 4 specifically built to interconnect MQTT brokers. Each broker runs inside a Docker container built on top of the HiveMQ CE image 5 .…”

E-MQTT: End-to-End Synchronous and Asynchronous Communication Mechanisms in MQTT Protocol

“…To facilitate a comprehensive evaluation, we drew a comparative analysis between the proposed E-MQTT QoS 3 and the existing MQTT QoS 2, given their closely aligned transmission processes. We implemented test clients (a publisher and subscribers) alongside broker applications that used both MQTT and E-MQTT because other benchmark frameworks or performance evaluation methods [56][57][58][59] do not support the features of E-MQTT. The source codes of the test applications (the broker, publisher, and subscriber) are available at https://github.com/ccslab/CM/tree/emqtt/CM/src (accessed on 14 November 2023).…”

“…Through the experiments, we measured five performance criteria: end-to-end delay, publish-completion delay, broker-subscriber delay, packet sizes, and energy consumption. They are normal metrics to evaluate the performance of MQTT-related research works [56][57][58][59]. Although we analyzed the number of exchanged messages in Section 3.1, the measurement of delay was necessary to figure out the overall overhead incurred by not only the number of messages but also other internal processing costs.…”