A scalable reliable instant messenger using the SD Erlang libraries

Abstract

Erlang has world leading reliability capabilities, but while it scales extremely well within a single node, distributed Erlang has some scalability issues. The Scalable Distributed (SD) Erlang libraries have been designed to address the scalability limitations while preserving the reliability model, and shown to deliver significant performance benefits above 40 hosts using some relatively simple benchmarks.This paper compares the reliability and scalability of SD Erlang and distributed Erlang using an Instant Messaging (IM) server benchmark that is a far more typical Erlang application; a relatively large and sophisticated benchmark; has throughput as the key performance metric; and uses non-trivial reliability mechanisms. We provide a careful reliability evaluation using chaos monkey.The key performance results consider scenarios with and without failures on up to 17 server hosts (272 cores). We show that SD Erlang adds no performance overhead when all nodes are grouped in a single s_group. However, either adding redundant router nodes in distributed Erlang applications, or dividing a set of nodes into small s_groups in SD Erlang applications, have small negative impact. Both the distributed Erlang and SD Erlang IM tolerate failures and, up to the failure rates measured, the failures have no impact on throughput. The IM implementations show that SD Erlang preserves the distributed Erlang reliability properties and mechanisms.

Scaling Reliably

“…those for managing node failures, network congestion, etc. A more detailed study of SD Erlang reliability, including the use of replicated databases for recovering Instant Messenger chat sessions, finds similar results [23].…”

“…For example, on a common X86/Ubuntu platform typical Erlang process recovery times are around 0.3ms, so around 1000x less than the Unix process recovery time on the same platform [58]. We have conducted more detailed experiments on an Instant Messenger benchmark, and obtained similar results [23].…”

“…The RELEASE project team also worked to improve the reliability and scalability of other Erlang programs including a substantial (approximately 150K lines of Erlang code) SimDiasca simulator [16] and an Instant Messenger that is more typical of Erlang applications [23] but we do not cover these systematically here.…”

Comparing languages for engineering server software

“…Comparing the performance of complete case study servers implemented in each language would significantly reinforce these results, and one possibility is an Instant Messaging (IM) benchmark [4]. It would also be interesting to study the performance overheads of providing fault tolerance and of recovering from faults, another key server capability.…”

“…The experiments are run on the Beowulf cluster (Section 4) with 16 physical (32 virtual) cores. We run the benchmark for 60s with different numbers of paired processes (1,2,4,6,8,16,32,64, and 128 pairs) varying the number of cores. The measurements reported are for cold starts, i.e.…”