PERVASIVE computing 75 80 PERVASIVE computing Presentation Input sensor Coordinator Model Figure 4. Application framework infrastructure. The coordinator oversees the composition of the model, presentation, and controller components.82 PERVASIVE computing PERVASIVE computing 83 the AUTHORS Manuel Román is a PhD candidate at the University of Illinois at Urbana-Champaign. His research interests include ubiquitous computing, middleware, operating systems, and interactive and programmable active spaces. He received his BS and MS in computer science from the La Salle School of Engineering (Ramon Llull Univ.).

Computational Science and Engineering (CSE) projects are typically developed by multidisciplinary teams. Despite being part of the same project, each team manages its own workflows, using specific execution environments and data processing tools. Analyzing the data processed by all workflows globally is a core task in a CSE project. However, this analysis is hard because the data generated by these workflows are not integrated. In addition, since these workflows may take a long time to execute, data analysis needs to be done at runtime to reduce cost and time of the CSE project. A typical solution in scientific data analysis is to capture and relate the data in a provenance database while the workflows run, thus allowing for data analysis at runtime. However, the main problem is that such data capture competes with the running workflows, adding significant overhead to their execution. To mitigate this problem, we introduce in this paper a system called ProvLake, which adopts design principles for providing efficient distributed data capture from the workflows. While capturing the data, ProvLake logically integrates and ingests them into a provenance database ready for analyses at runtime. We validated ProvLake in a real use case in the O&G industry encompassing four workflows that process 5 TB datasets for a deep learning classifier. Compared with Komadu, the closest solution that meets our goals, our approach enables runtime multiworkflow data analysis with much smaller overhead, such as 0.1%. Geological raw data files Kubernetes VolumeInter-workflow Data RelationshipsMulti-store Data Relationships Legend Deep learning training datasets

Machine Learning (ML) has become essential in several industries. In Computational Science and Engineering (CSE), the complexity of the ML lifecycle comes from the large variety of data, scientists' expertise, tools, and workflows. If data are not tracked properly during the lifecycle, it becomes unfeasible to recreate a ML model from scratch or to explain to stakeholders how it was created. The main limitation of provenance tracking solutions is that they cannot cope with provenance capture and integration of domain and ML data processed in the multiple workflows in the lifecycle, while keeping the provenance capture overhead low. To handle this problem, in this paper we contribute with a detailed characterization of provenance data in the ML lifecycle in CSE; a new provenance data representation, called PROV-ML, built on top of W3C PROV and ML Schema; and extensions to a system that tracks provenance from multiple workflows to address the characteristics of ML and CSE, and to allow for provenance queries with a standard vocabulary. We show a practical use in a real case in the O&G industry, along with its evaluation using 48 GPUs in parallel.Index Terms-Machine Learning Lifecycle, Workflow Provenance, Computational Science and Engineering (ii) PROV-ML: a new data representation, which combines W3C PROV [18] with W3C ML Schema [19], for prove-R. Souza et al. Provenance Data in the Machine Learning Lifecycle in Computational Science and Engineering.

No abstract