Verifying Concurrent Multicopy Search Structures

Abstract

Multicopy search structures such as log-structured merge (LSM) trees are optimized for high insert/update/delete (collectively known as upsert) performance. In such data structures, an upsert on key 𝑘, which adds (𝑘, 𝑣) where 𝑣 can be a value or a tombstone, is added to the root node even if 𝑘 is already present in other nodes. Thus there may be multiple copies of 𝑘 in the search structure. A search on 𝑘 aims to return the value associated with the most recent upsert. We present a general framework for verifying linearizability of concurrent multicopy search structures that abstracts from the underlying representation of the data structure in memory, enabling proof-reuse across diverse implementations. Based on our framework, we propose template algorithms for a) LSM structures forming arbitrary directed acyclic graphs and b) differential file structures, and formally verify these templates in the concurrent separation logic Iris. We also instantiate the LSM template to obtain the first verified concurrent in-memory LSM tree implementation.