WiSer: A Highly Available HTAP DBMS for IoT Applications

Abstract

In a classic transactional distributed database management system (DBMS), write transactions invariably synchronize with a coordinator before final commitment. While enforcing serializability, this model has long been criticized for not satisfying the applications' availability requirements. When entering the era of Internet of Things (IoT), this problem has become more severe, as an increasing number of applications call for the capability of hybrid transactional and analytical processing (HTAP), where aggregation constraints need to be enforced as part of transactions. Current systems work around this by creating escrows, allowing occasional overshoots of constraints, which are handled via compensating application logic.The WiSer DBMS targets consistency with availability, by splitting the database commit into two steps. First, a PROMISE step that corresponds to what humans are used to as commitment, and runs without talking to a coordinator. Second, a SERIALIZE step, that fixes transactions' positions in the serializable order, via a consensus procedure. We achieve this split via a novel data representation that embeds read-sets into transaction deltas, and serialization sequence numbers into table rows. WiSer does no sharding (all nodes can run transactions that modify the entire database), and yet enforces aggregation constraints. Both read-write conflicts and aggregation constraint violations are resolved lazily in the serialized data. WiSer also covers node joins and departures as database tables, thus simplifying correctness and failure handling. We present the design of WiSer as well as experiments suggesting this approach has promise.

A survey on hybrid transactional and analytical processing

“…Wiser [21] seeks consensus protocols to achieve the serial order of writes. A query can only see updates made before it in a serializable order generated by the consensus (i.e., Raft [125]).…”

Heterogeneous Data-Centric Architectures for Modern Data-Intensive Applications: Case Studies in Machine Learning and Databases

“…Many application domains, such as fraud detection [169][170][171], business intelligence [172][173][174], healthcare [175,176], personalized recommendation [177,178], and IoT [177], have a critical need to perform real-time data analysis, where data analysis needs to be performed using the most recent version of data [179,180]. To enable real-time data analysis, stateof-the-art database management systems (DBMSs) leverage hybrid transactional and analytical processing (HTAP) [181][182][183].…”

Enabling High-Performance and Energy-Efficient Hybrid Transactional/Analytical Databases with Hardware/Software Cooperation

“…Data analytics has become popular due to the rapid growth of data generated annually [1]. Many application domains, such as fraud detection [2][3][4], business intelligence [5][6][7], healthcare [8,9], personalized recommendation [10,11], and IoT [10], have a critical need to perform real-time data analysis, where data analysis needs to be performed using the most recent version of data [12,13]. To enable real-time data analysis, state-of-the-art database management systems (DBMSs) leverage hybrid transactional and analytical processing (HTAP) [14][15][16].…”