The Best of Both Worlds: Challenges in Linking Provenance and Explainability in Distributed Machine Learning

Abstract

Machine learning experts prefer to think of their input as a single, homogeneous, and consistent data set. However, when analyzing large volumes of data, the entire data set may not be manageable on a single server, but must be stored on a distributed file system instead. Moreover, with the pressing demand to deliver explainable models, the experts may no longer focus on the machine learning algorithms in isolation, but must take into account the distributed nature of the data stored, as well as the impact of any data pre-processing steps upstream in their data analysis pipeline. In this paper, we make the point that even basic transformations during data preparation can impact the model learned, and that this is exacerbated in a distributed setting. We then sketch our vision of end-to-end explainability of the model learned, taking the pre-processing into account. In particular, we point out the potentials of linking the contributions of research on data provenance with the efforts on explainability in machine learning. In doing so, we highlight pitfalls we may experience in a distributed system on the way to generating more holistic explanations for our machine learning models.

Provenance Supporting Hyperparameter Analysis in Deep Neural Networks

“…With the recent interest in DL methods, several works propose provenance management approaches for data analysis during DNN training [11]. There are several challenges in making ML workflows provenance aware like taking into account the execution framework that may involve CPUs, GPUs, TPUs, and distributed environments such as clusters and clouds as discussed in [36,42,14]. In this section, we discuss related work for provenance data management, considering the intention of using provenance for runtime data analysis.…”

“…The approaches in this category manage provenance for several purposes in ML platforms [46,35,24,40,2,36,30,41,12]. They are all based on a proprietary representation of provenance data, i.e., that does not follow recommendations like W3C PROV.…”

“…Basically, there are two approaches to make ML workflows provenance aware. The first is provenance provided for a specific ML platform [46,35,24,40,2,20,36,30] and the second is the provenance systems that are independent of the domain [32]. In the first approach, each ML platform provides provenance using its proprietary representation, which is difficult to interpret and compare with execution between different platforms.…”

Capturing and querying fine-grained provenance of preprocessing pipelines in data science

“…Specifically, a number of tools are available to help developers build machine learning pipelines [50,18,51] or debug them [52], but these lack the ability to explain the provenance of a certain data item in the processed dataset. Others link provenance to explainability in a distributed machine learning setting [53] but without offering specific tools. Amazon identifies that there are common and reusable components to a machine learning pipeline, but that there is no way to track the exploration of pipeline construction effectively, and calls for metadata capture to support reasoning over pipeline design [54].…”

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

“…To explain how a DML algorithm gives a decision, all transformations applied to the data should be considered. It is claimed in [214] that even basic transformations in data pre-processing, such as data partitioning, local data cleaning and value imputation, can have a strong impact on the resultant model. The effect becomes more apparent under a distributed setting.…”