SIGNORA: A Blockchain-Based Framework for Dataflow Integrity Provisioning in an Untrusted Data Pipeline

Abstract

In a typical data pipeline, the dataflow starts from the first node, where the data is initiated and moves to the last node in the pipeline, where the processed data will be stored. Due to the sheer number of the involved participants, it is crucial to protect the dataflow integrity in the pipeline. While previous studies have outlined solutions to this matter, the solution for an untrusted data pipeline is still left unexplored, which motivates us to propose SIGNORA. Our proposal combines the concept of a chain of signatures with blockchain receipt to provide dataflow integrity in a data pipeline. The chain of signatures provides a non-repudiation guarantee from participants, while the hash of the data and signatures is anchored in the blockchain for a non-tampering guarantee through blockchain receipt. Aside from that, SIGNORA also satisfies essential requirements of running data pipeline processing in an open and untrusted environment, such as (i) providing reliable identity management, (ii) solving the trust and accountability issues through a reputation system, and (iii) supporting various devices through multiple cryptographic algorithms (i.e., ECDSA, EdDSA, RSA, and HMAC) and (iv) off-chain processing. Our experiment results show that SIGNORA can provide dataflow integrity provisioning in multiple scenarios of data payload size with reasonable overhead. Furthermore, the cost of smart contract methods has also been analyzed, and several off-chain solutions have been addressed to reduce transaction costs. Finally, the reputation system can adapt to the history of nodes' activities by increasing their scores when they actively perform honest behavior while reducing their scores when they become inactive. Therefore, SIGNORA can provide a high degree of accountability for participants collaborating in an untrusted environment.