CEC: Continuous eventual checkpointing for data stream processing operators

Sebepou, Zoe; Magoutis, Kostas

doi:10.1109/dsn.2011.5958214

Cited by 36 publications

(32 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If states are small in space and if event processing is costly, this efficiently recovers a correct state. However, for large detector states [Sebepou and Magoutis 2011;Arasu et al 2004] rollback recovery becomes infeasible, as the memory needed to store the states outgrows the available system memory. In those cases, savepoints [Koldehofe et al 2013] significantly outperform approaches that are based on persistent checkpoints.…”

Section: Reliable Event Processingmentioning

confidence: 99%

Adaptive Speculative Processing of Out-of-Order Event Streams

Mutschler

Philippsen

2014

ACM Trans. Internet Technol.

View full text Add to dashboard Cite

Distributed event-based systems are used to detect meaningful events with low latency in high data-rate event streams that occur in surveillance, sports, finances, etc. However, both known approaches to dealing with the predominant out-of-order event arrival at the distributed detectors have their shortcomings: buffering approaches introduce latencies for event ordering, and stream revision approaches may result in system overloads due to unbounded retraction cascades.This article presents an adaptive speculative processing technique for out-of-order event streams that enhances typical buffering approaches. In contrast to other stream revision approaches developed so far, our novel technique encapsulates the event detector, uses the buffering technique to delay events but also speculatively processes a portion of it, and adapts the degree of speculation at runtime to fit the available system resources so that detection latency becomes minimal.Our technique outperforms known approaches on both synthetical data and real sensor data from a realtime locating system (RTLS) with several thousands of out-of-order sensor events per second. Speculative buffering exploits system resources and reduces latency by 40% on average. ACM Reference Format:Christopher Mutschler and Michael Philippsen. 2014. Adaptive speculative processing of out-of-order event streams.

show abstract

Section: Reliable Event Processingmentioning

confidence: 99%

Adaptive Speculative Processing of Out-of-Order Event Streams

Mutschler

Philippsen

2014

ACM Trans. Internet Technol.

View full text Add to dashboard Cite

show abstract

“…There are various definitions to what fault tolerance is. In dealing with fault tolerance, replication is typically used for general fault tolerance method to protect against system failure [1] [2]. Sebepou et al highlighted three major forms of replication mechanism which are [1] [2]:…”

Section: Introductionmentioning

confidence: 99%

Fault Tolerance Mechanisms in Distributed Systems

Akkaya¹

2015

IJCNS

View full text Add to dashboard Cite

The use of technology has increased vastly and today computer systems are interconnected via different communication medium. The use of distributed systems in our day to day activities has solely improved with data distributions. This is because distributed systems enable nodes to organise and allow their resources to be used among the connected systems or devices that make people to be integrated with geographically distributed computing facilities. The distributed systems may lead to lack of service availability due to multiple system failures on multiple failure points. This article highlights the different fault tolerance mechanism in distributed systems used to prevent multiple system failures on multiple failure points by considering replication, high redundancy and high availability of the distributed services.

show abstract

“…To reduce recovery times in passive replication, Sebepou et al [28] partition the state into smaller chunks, updating operator state incrementally. Their approach is only evaluated for aggregate operators, and it remains unclear how it can be applied to other types of stateful partitioned operators.…”

Section: Related Workmentioning

confidence: 99%

“…While mechanisms for scale out [27,26] and fault tolerance [30,28,33] in stream processing have received considerable attention in the past, it remains an open question how SPSs can scale out while remaining fault tolerant when queries contain stateful operators. Especially with recently popular stream processing models [23,29] that treat operators as black boxes in a data flow graph, users rely on operators that have large amounts of state, which potentially depends on the complete history of previously processed tuples [5].…”

Section: Introductionmentioning

confidence: 99%