60 Years of Mastering Concurrent Computing through Sequential Thinking

Abstract

Modern computing systems are highly concurrent. Threads run concurrently in shared-memory multi-core systems, and programs run in different servers communicating by sending messages to each other. Concurrent programming is hard because it requires to cope with many possible, unpredictable behaviors of the processes, and the communication media. The article argues that right from the start in 1960's, the main way of dealing with concurrency has been by reduction to sequential reasoning. It traces this history, and illustrates it through several examples, from early ideas based on mutual exclusion (which was initially introduced to access shared physical resources), passing through consensus and concurrent objects (which are immaterial data), until today distributed ledgers. A discussion is also presented, which addresses the limits that this approach encounters, related to fault-tolerance, performance, and inherently concurrent problems.

FIFO and Atomic broadcast algorithms with bounded message size for dynamic systems

“…An extensive survey, collecting around sixty total order broadcast algorithms in message passing model, is presented in [3] where the algorithms are classified according to the mechanism they use to order messages: communication history, privilege-based, moving sequencer, fixed sequencer, and destinations agreement. It is worth pointing out that total order broadcast primitives are a fundamental building block for a lot of distributed systems, applications, algorithms and services such as State Machine Replication [20], managing replicated database [21], [22], [23], sequential consistency [24], [25], shared registers implementation with Byzantine failure [26] or continuous churns [27].…”

“…At every update, s increments myCnt value (line 16). It is reinitialized to 0 at the beginning of each broadcast (lines [19][20][21][22][23][24]. The label and updtCnt values of messages are used by a process r = s to choose which messages of s to keep in its set M sgsT (r) -(if-condition at line 8).…”

Loose Coupling: An Invisible Thread in the History of Technology

“…The paper [79] describes one of the first distributed computing systems, but Dijkstra had started the whole field already in 1965 [80]. The opposite of distributed computing is local computing.…”

Assessing Distributed Consensus Performance on Mobile Cyber-Physical System Swarms

“…In this context, distributed consensus protocols contribute to enabling swarm intelligence by providing fault-tolerant sub-systems including dynamic reconfiguration, distributed locking, reliable broadcast and state machine replication. Although consensus protocols have been extensively studied [9], [12], [20], [22], [30], [32], [37], [43], [44], previous works mostly overlooked the performance evaluation and practical aspects of consensus algorithms in mobile systems in general and in mobile CPS swarms in particular [35]. For instance, it is well known that wireless communication eventually increases message loss, and that network partitions of the communication network are far more likely to occur in mobile environments [45].…”