Drop the anchor

Verification of concurrent data structures is one of the most challenging tasks in software verification. The topic has received considerable attention over the course of the last decade. Nevertheless, human-driven techniques remain cumbersome and notoriously difficult while automated approaches suffer from limited applicability. The main obstacle for automation is the complexity of concurrent data structures. This is particularly true in the absence of garbage collection. The intricacy of lock-free memory management paired with the complexity of concurrent data structures makes automated verification prohibitive.In this work we present a method for verifying concurrent data structures and their memory management separately. We suggest two simpler verification tasks that imply the correctness of the data structure. The first task establishes an over-approximation of the reclamation behavior of the memory management. The second task exploits this over-approximation to verify the data structure without the need to consider the implementation of the memory management itself. To make the resulting verification tasks tractable for automated techniques, we establish a second result. We show that a verification tool needs to consider only executions where a single memory location is reused. We implemented our approach and were able to verify linearizability of Michael&Scott's queue and the DGLM queue for both hazard pointers and epoch-based reclamation. To the best of our knowledge, we are the first to verify such implementations fully automatically.Otherwise, this equality may not hold.Property (G5) holds by definition together with Property (P5). It remains to show Properties (G2) to (G4) and (G6).Ad Property (G2). First consider the case where we have q ∈ valid τ . By Auxiliary (A1) we also have q ∈ valid σ . We getThe second equality holds because m τ (q) = b. That is, we preserve b.data in m τ | valid τ and only need to update the mapping of p. Similarly, we getNow consider the case q valid τ . By Auxiliary (A1) we also have q valid σ . We getThe last equality holds because the update does not survive the restriction to a set which is guaranteed not to contain p. Similarly, we getWe now get:The first equality is by definition of restrictions, the second equality is due to Property (P2), the third one is by Auxiliary (A1), and the last is again by definition. This concludes the property.Ad Property (G3). Only the valuation of p is changed by both act and act ′ . So by Property (P3) it suffices to show m τ .act (p)=a ⇐⇒ m σ .act ′ (p)=a. This follows easily:where the first and last equivalence hold due to the updates up and up ′ and the second equality holds by Property (P3).Ad Property (G4). Let c ∈ m τ .act (valid τ .act ). We have c ∈ m τ (valid τ ). To see this, consider some pexp ∈ valid τ .act such that m τ .act (pexp) = c. If pexp p, then m τ .act (pexp) = m τ (pexp) and pexp ∈ valid τ by definition. So c ∈ m τ (valid τ ) follows immediately. Otherwise, in the case of pexp ≡ p, we have m τ .act (p) = m τ (q). ...

show abstract

“…Drop the Anchor (DTA) [Braginsky et al 2013]. A version of EBR which uses HP to fight thread failures.…”

Section: B Extended Discussion Of Smr Algorithmsmentioning

confidence: 99%

Decoupling lock-free data structures from memory reclamation for static analysis

Meyer

Wolff

2019

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

show abstract

“…Incorporating a manual scheme into a lock-free data structure implies protecting the object (node) before accessing it, and retiring the object when it is no longer reachable. In lock-free schemes [5,7,11,14,19,24,25], this implies calling a protection method for a pointer so as to read it safely from a given memory address, get_protected(addr), before dereferencing the pointer to access variables in the object, and calling the retire(ptr) when the object is no longer reachable from the root pointers of the data structure. Determining where to call these two methods on a lock-free data structure, can be a challenging task, requiring significant expertise.…”

Section: Related Workmentioning

confidence: 99%

“…Drop-the-anchor (DTA) [5] extends HP and also improves the performance of the protect functionality. Applying DTA to a lock-free data structure may require non-trivial modifications to the data structure's algorithm.…”

Section: Related Workmentioning

confidence: 99%

“…Lock-free data structure algorithms cannot utilize lock-based GCs if they wish to keep the lock-free progress they were designed for. Until now, for most data structures, lock-free manual memory reclamation schemes [5,11,14,19,25] have been the only option. Nevertheless, incorporating these lock-free memory reclamation schemes on lock-free data structure algorithms remains a complex task, shun by most data structure designers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

OrcGC

Correia

Ramalhete

Felber

2021

Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

View full text Add to dashboard Cite

Dynamic lock-free data structures require a memory reclamation scheme with a similar progress. Until today, lock-free schemes are applied to data structures on a case-by-case basis, often with algorithm modifications to the data structure.In this paper we introduce two new lock-free reclamation schemes, one manual and the other automatic with user annotated types. The manual reclamation scheme, named pass-the-pointer (PTP), has lock-free progress and a bound on the number of unreclaimed objects that is linear with the number of threads.The automatic lock-free memory reclamation scheme, which we named OrcGC, uses PTP and object reference counting to automatically detect when to protect and when to de-allocate an object. OrcGC has a linear bound on memory usage and supports the system allocator. We propose a new methodology that utilizes OrcGC to provide lock-free memory reclamation to a data structure.We conducted a performance evaluation on two machines, an Intel and an AMD, applying PTP and OrcGC to several lock-free data structures, providing lock-free memory reclamation where before there was none. On the Intel machine we saw no significant performance impact, while on AMD we observed a worst-case performance drop below 50%.CCS Concepts • Theory of computation Concurrent algorithms.

show abstract

“…Dynamic Collect [Dragojevic et al 2011], StackTrack [Alistarh et al 2014], and ThreadScan [Alistarh et al 2015] are HP-esque implementations exploring the use of operating system and hardware support. Drop the Anchor [Braginsky et al 2013], Optimistic Access [Cohen and Petrank 2015b], Automatic Optimistic Access [Cohen and Petrank 2015a], QSense [Balmau et al 2016], Hazard Eras [Ramalhete and Correia 2017], and Interval-based Reclamation [Wen et al 2018] combine EBR and HP. Free Access [Cohen 2018] automates the application of Automatic Optimistic Access.…”

Section: Related Workmentioning

confidence: 99%

Pointer life cycle types for lock-free data structures with memory reclamation

Meyer

Wolff

2019

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

We consider the verification of lock-free data structures that manually manage their memory with the help of a safe memory reclamation (SMR) algorithm. Our first contribution is a type system that checks whether a program properly manages its memory. If the type check succeeds, it is safe to ignore the SMR algorithm and consider the program under garbage collection. Intuitively, our types track the protection of pointers as guaranteed by the SMR algorithm. There are two design decisions. The type system does not track any shape information, which makes it extremely lightweight. Instead, we rely on invariant annotations that postulate a protection by the SMR. To this end, we introduce angels, ghost variables with an angelic semantics. Moreover, the SMR algorithm is not hard-coded but a parameter of the type system definition. To achieve this, we rely on a recent specification language for SMR algorithms. Our second contribution is to automate the type inference and the invariant check. For the type inference, we show a quadratic-time algorithm. For the invariant check, we give a source-to-source translation that links our programs to off-the-shelf verification tools. It compiles away the angelic semantics. This allows us to infer appropriate annotations automatically in a guess-and-check manner. To demonstrate the effectiveness of our type-based verification approach, we check linearizability for various list and set implementations from the literature with both hazard pointers and epoch-based memory reclamation. For many of the examples, this is the first time they are verified automatically. For the ones where there is a competitor, we obtain a speed-up of up to two orders of magnitude.Proving lock-free data structures linearizable has received a lot of attention (cf. Section 10). Doherty et al. [2004], for instance, give a mechanized proof of a practical lock-free queue. Such proofs require plenty of manual work and take a considerable amount of time. Moreover, they require an understanding of the proof method and the data structure under consideration. To overcome this drawback, we are interested in automated verification. The cave tool by Vafeiadis [2010a,b], for example, is able to establish linearizability for singly-linked data structures fully automatically.The problem with automated verification for lock-free data structures is its limited applicability. Most techniques are restricted to implementations that assume a garbage collector (GC). This assumption, however, does not apply to all programming languages. Take C/C++ as an example. It does not provide an automatic garbage collector that is running in the background. Instead, it is the programmer's obligation to avoid memory leaks by reclaiming memory that is no longer in use (using delete). In lock-free data structures, this task is much harder than it may seem at first glance. The root of the problem is that threads typically traverse the data structure without synchronization. Hence, there may be threads holding pointers to records that have alread...

show abstract

Drop the anchor

Cited by 45 publications

References 18 publications

Decoupling lock-free data structures from memory reclamation for static analysis

Decoupling lock-free data structures from memory reclamation for static analysis

OrcGC

Pointer life cycle types for lock-free data structures with memory reclamation

Contact Info

Product

Resources

About