A conceptual architecture for contractual data sharing in a decentralised environment

Abstract

Machine Learning systems rely on data for training, input and ongoing feedback and validation. Data in the field can come from varied sources, often anonymous or unknown to the ultimate users of the data. Whenever data is sourced and used, its consumers need assurance that the data accuracy is as described, that the data has been obtained legitimately, and they need to understand the terms under which the data is made available so that they can honour them. Similarly, suppliers of data require assurances that their data is being used legitimately by authorised parties, in accordance with their terms, and that usage is appropriately recompensed. Furthermore, both parties may want to agree on a specific set of quality of service (QoS) metrics, which can be used to negotiate service quality based on cost, and then receive affirmation that data is being supplied within those agreed QoS levels. Here we present a conceptual architecture which enables data sharing agreements to be encoded and computationally enforced, remuneration to be made when required, and a trusted audit trail to be produced for later analysis or reproduction of the environment. Our architecture uses blockchainbased distributed ledger technology, which can facilitate transactions in situations where parties do not have an established trust relationship or centralised command and control structures. We explore techniques to promote faith in the accuracy of the supplied data, and to let data users determine trade-offs between data quality and cost. Our system is exemplified through consideration of a case study using multiple data sources from different parties to monitor traffic levels in urban locations.

Providing assurance and scrutability on shared data and machine learning models with verifiable credentials

“…In previous work, 30 the authors have proposed a model of a supply chain for AI/ML products, based on the premise of ML model authors and publishers populating a BOM document that provides details of the contributions of components, such as data sources and other artifacts, along with the human work toward creation of the resultant ML model. Describing ML systems in terms of their supply chain provides a mechanism to identify data sources and assets which contribute to the development of these data components.…”

A framework for fostering transparency in shared artificial intelligence models by increasing visibility of contributions

“…Future research will attempt to identify a set of objective measures that will be capable of being mechanised to replace the subjective elements of the ranking. A Bill of Materials document supplied or made available with the model, as proposed 43 , would be a suitable vehicle for making such information about the contributors available to model users, as it facilitates both the identification of significant contributions to the system (the nodes) as well as providing a means to identify supporting information and artifacts. Other proposals for documentation of ML systems, such as the Model Card proposal from Mitchell et al 27 or a Supplier's Declaration of Conformity as suggested by Hind, et al 30 , could also be used as the basis for a framework which delivers information from which the visibility judgements could be made.…”

Quantifying Transparency of Machine Learning Systems through Analysis of Contributions

“…Future research will attempt to identify a set of objective measures that will be capable of being mechanised to replaces the subjective elements of the ranking. A Bill of Materials document supplied or made available with the model, as proposed [2], would be a suitable vehicle for making such information about the contributors available to model users, as it facilitates both the identification of significant contributions to the system (the nodes) as well as providing a means to identify supporting information and artifacts. Other proposals for documentation of ML systems, such as the Model Card proposal from Mitchell et al [19] or a Supplier's Declaration of Conformity as suggested by Hind, et al [20], could also be used to provide information from which the visibility judgements could be made.…”

“…In considering the development pipelines for ML models we can identify the contributing assets, which will typically include data sets used for training and validation, and human expertise which is used both in the preparation and curation of training data, and in the development and calibration of the resultant model. In previous work [1], [2] we have considered the benefits of applying established techniques from industry and agri-food to provide transparency and traceability on contributions to data products created through the aggregation of multiple machine and human input sources, including supply chain modelling (SCM) and the maintenance of a Bill of Materials (BoM) document to clearly identify the contributors to the output products of data ecosystems and machine learning pipelines.…”