Verifying Pipelined-RAM Consistency over Read/Write Traces of Data Replicas

Wei, Hengfeng; Biasi, Marzio De; Huang, Yu; Cao, Jiannong; Lü, Jian

doi:10.1109/tpds.2015.2453985

Cited by 5 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the testing problem of relaxed memory consistency models, Wei et al [18] proved that complexity of testing PRAM consistency is NP-complete. Bouajjani et al [19] studied the complexity of verifying causal consistency for one history.…”

Section: Undecidability Of Verifying Weak Consistencymentioning

confidence: 99%

The Complexity of Weak Consistency

Liu

2018

Frontiers in Algorithmics

View full text Add to dashboard Cite

Weak consistency is a memory model that is frequently considered for shared memory systems. Its most distinguishable feature lies in a category of operations in two types: data operations and synchronization operations. For highly parallel shared memory systems, this model offers greater performance potential than strong models such as sequential consistency by permitting unconstrained optimization on updates propagation before synchronization is invoked. It captures the intuition that delaying updates produced by data operations before triggering a synchronization operation does not typically affect the program correctness. To formalize the connection between concrete executions and the corresponding specification, we propose in this work a new approach to define weak consistency. This formalization, defined in terms of distributed histories abstracted from concrete executions, provides an additional perception of the concept and facilitates automatic analysis of system behaviors. We then investigate the problems on verifying whether implementations have correctly implemented weak consistency. Specifically, we consider two problems: (1) the testing problem that checks whether one single execution is weakly consistent, a critical problem for designing efficient testing and bug hunting algorithms, and (2) the model checking problem that determines whether all executions of an implementation are weakly consistent. We show that the testing problem is NP-complete, even for finite processes and short programs. The model checking problem is proven to be undecidable.

show abstract

Section: Undecidability Of Verifying Weak Consistencymentioning

confidence: 99%

The Complexity of Weak Consistency

Liu

2018

Frontiers in Algorithmics

View full text Add to dashboard Cite

show abstract

“…Wei et al [41] studied the complexity of verifying PRAM consistency (also called FIFO consistency) for one history. They proved that the problem is NP-complete.…”

Section: Related Workmentioning

confidence: 99%

On verifying causal consistency

BouajjaniAhmed¹,

EneaConstantin²,

GuerraouiRachid³

et al. 2017

SIGPLAN Not.

View full text Add to dashboard Cite

Abstract. Causal consistency is one of the most adopted consistency criteria for distributed implementations of data structures. It ensures that operations are executed at all sites according to their causal precedence. We address the issue of verifying automatically whether the executions of an implementation of a data structure are causally consistent. We consider two problems: (1) checking whether one single execution is causally consistent, which is relevant for developing testing and bug finding algorithms, and (2) verifying whether all the executions of an implementation are causally consistent.We show that the first problem is NP-complete. This holds even for the read-write memory abstraction, which is a building block of many modern distributed systems. Indeed, such systems often store data in key-value stores, which are instances of the readwrite memory abstraction. Moreover, we prove that, surprisingly, the second problem is undecidable, and again this holds even for the read-write memory abstraction. However, we show that for the read-write memory abstraction, these negative results can be circumvented if the implementations are data independent, i.e., their behaviors do not depend on the data values that are written or read at each moment, which is a realistic assumption.We prove that for data independent implementations, the problem of checking the correctness of a single execution w.r.t. the read-write memory abstraction is polynomial time. Furthermore, we show that for such implementations the set of non-causally consistent executions can be represented by means of a finite number of register automata. Using these machines as observers (in parallel with the implementation) allows to reduce polynomially the problem of checking causal consistency to a state reachability problem. This reduction holds regardless of the class of programs used for the implementation, of the number of read-write variables, and of the used data domain. It allows leveraging existing techniques for assertion/reachability checking to causal consistency verification. Moreover, for a significant class of implementations, we derive from this reduction the decidability of verifying causal consistency w.r.t. the read-write memory abstraction.

show abstract

“…To face different scale of data and user workloads, cloud providers offer virtual machines (VMs) based elastic Hadoop applications. For example, Amazon EC2 provides Amazon EMR 2) , which allows tenants to run big data applications in the cloud. OpenStack foundation provides Sahara 3) to host the data-intensive application cluster easily.…”

Section: Introductionmentioning

confidence: 99%

“…Supported by the HDFS [1], the Hadoop applications are data-intensive. Thus, apart from guaranteeing the data consistency [2,3], improving IO performance will help reduce the JCT.…”

Section: Introductionmentioning

confidence: 99%

IO dependent SSD cache allocation for elastic Hadoop applications

Tang¹,

Wang²,

Sun³

et al. 2018

Sci. China Inf. Sci.

Self Cite

View full text Add to dashboard Cite

Elastic Hadoop applications consisting of multiple virtual machines (VMs) are widely used to support big data analysis and processing. In this scenario, flash-based solid state drive (SSD) is usually deployed on hypervisors and used as the cache to improve the IO performance. However, existing SSD caching schemes are mostly VM-centric, which focus on the low-level IO performance metrics of individual VMs. They may not lead to the optimized performance of elastic Hadoop applications, i.e., the job completion time (JCT), as the importance of VMs inside the application are different even though they have the similar low-level IO patterns. Considering the IO dependency among VMs and figuring out the importance, which we regard as the application-centric metrics, may potentially better improve the performance. We present IO dependency based requirement model, to characterize the requirement of SSD cache for each VM inside the elastic Hadoop application, and then use it in a genetic algorithm (GA) based approach to calculate the nearly optimal weights of VMs for allocating the per-VM SSD cache space and the capacity of the I/O operations per second (IOPS). Furthermore, we present a tool AC-SSD based on the approach and introduce the closed-loop adaptation to react to continuously changing workloads. The evaluation shows that by using AC-SSD, the JCT is reduced by up to 39% for IO sensitive workloads, up to 29% for continuously changing workloads, and over 12.5% for different scale of data comparing to the shared cache.

show abstract

Verifying Pipelined-RAM Consistency over Read/Write Traces of Data Replicas

Cited by 5 publications

References 25 publications

The Complexity of Weak Consistency

The Complexity of Weak Consistency

On verifying causal consistency

IO dependent SSD cache allocation for elastic Hadoop applications

Contact Info

Product

Resources

About