Assessing and restoring reproducibility of Jupyter notebooks

Wang, Jiawei; Kuo, Tzu-yang; Li, Li; Zeller, Andreas

doi:10.1145/3324884.3416585

Cited by 49 publications

(28 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Out [4]: co = 0 Seguir algumas dessas boas práticas manualmente pode exigir um esforço cognitivo de estar sempre organizando o notebook. Algumas ferramentas foram propostas para diminuir esse esforço e melhorar a qualidade e reprodutibilidade de notebooks: Julynter , NBSafety [Macke et al 2020], ReproZip [Chirigati et al 2016], nbgather [Head et al 2019] e Osiris [Wang et al 2020].…”

Section: Reprodução E Boas Práticasunclassified

“…Por fim, muitas vezes essas ferramentas não foram usadas durante o desenvolvimento e é necessário reproduzir notebooks seguindo a ordem correta. Para resolver esses problemas, Osiris [Wang et al 2020] busca reordenar as células de um notebook para uma ordem reprodutível. Para isso, Osiris analisa a ocorrência de definição e uso de nomes em cada célula do notebook e reordena a execução de forma a minimizar a presença de células fora de ordem.…”

Section: Reprodução E Boas Práticasunclassified

See 1 more Smart Citation

Ciência de Dados com Reprodutibilidade usando Jupyter

Pimentel¹,

Oliveira²,

Silva³

et al. 2021

Jornada De Atualização Em Informática 2021

View full text Add to dashboard Cite

Data Science has become a trending research topic in Computer Science due to the growing interest in extracting knowledge from different data sources. In such a context, Jupyter Notebook has consolidated itself as one of the main tools used by data scientists to perform exploratory data analysis in a fast and straightforward way, with a high potential for code reproduction. Hence, this JAI aims to present Jupyter with reproducibility for developing Data Science projects. The content is tailored for students and professionals with some programming experience. In particular, we first introduce Jupyter and its general use to develop solutions for Data Science. Then, we present Jupyter advanced topics and address ways to promote open science. Finally, this JAI overviews Data Science with Jupyter Notebooks by combining concepts and theoretical foundations with practical examples and real-world data. ResumoCiência de Dados tornou-se um tópico de pesquisa emergente na Ciência da Computação devido ao crescente interesse em extrair conhecimento de diferentes fontes de dados. Nesse contexto, o Jupyter Notebook vem se consolidando como uma das principais ferramentas utilizadas por cientistas de dados para realizar análises exploratórias de dados de forma rápida e direta, com alto potencial de reprodução de código. Dessa forma, o objetivo deste capítulo é apresentar o Jupyter com reprodutibilidade para a realização de projetos em Ciência de Dados. O conteúdo é organizado para estudantes e profissionais com alguma experiência em programação. Em particular, primeiro apresentamos o Jupyter e seu uso geral para desenvolver soluções para Ciência de Dados. Em seguida, apresentamos tópicos avançados do Jupyter e abordamos maneiras de promover a ciência aberta. Para concluir, este JAI apresenta uma visão geral de Ciência de Dados com Jupyter Notebooks combinando conceitos e fundamentos teóricos com exemplos práticos e dados do mundo real.

show abstract

Section: Reprodução E Boas Práticasunclassified

Ciência de Dados com Reprodutibilidade usando Jupyter

Pimentel¹,

Oliveira²,

Silva³

et al. 2021

Jornada De Atualização Em Informática 2021

View full text Add to dashboard Cite

show abstract

“…This is because when code cells are repeatedly executed, the original execution counter will be overwritten by the last execution counter-called a skip of execution counter. Skips make it hard to reproduce the original outputs of a notebook because the skipped execution counters are not recorded at all [2].…”

Section: B Jupyter Notebooksmentioning

confidence: 99%

“…In the last research question, we evaluate if SnifferDog helps users not only execute, but reproduce Jupyter notebooks (i.e., achieving the same outputs as that recorded originally in the notebooks). To resolve the problem of cell ordering, we make use of a prototype tool called Osiris that restores the order in which cells are to be executed [2].…”

Section: Rq6: Reproducing Jupyter Notebooksmentioning

confidence: 99%

“…However, there are also reasons for notebooks being non-executable that can be easily avoided. o n e reason is that notebook code cells can be executed interactively in any order (and data scientists happily do so); recent approaches [2] thus focus on restoring the actual order based on internal dependencies. Another important reason, however, is that Jupyter notebooks depend on specific environments in which they were created, such as specific libraries in specific versions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Restoring Execution Environments of Jupyter Notebooks

Wang

Zeller

2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

Self Cite

View full text Add to dashboard Cite

More than ninety percent of published Jupyter notebooks do not state dependencies on external packages. This makes them non-executable and thus hinders reproducibility of scientific results. We present SnifferDog, an approach that 1) collects the APIs of Python packages and versions, creating a database of APIs; 2) analyzes notebooks to determine candidates for required packages and versions; and 3) checks which packages are required to make the notebook executable (and ideally, reproduce its stored results). In its evaluation, we show that SnifferDog precisely restores execution environments for the largest majority of notebooks, making them immediately executable for end users.

show abstract

Notebook‐as‐a‐VRE (NaaVRE): From private notebooks to a collaborative cloud virtual research environment

et al. 2022

View full text Add to dashboard Cite

Virtual research environments (VREs) provide user‐centric support in the lifecycle of research activities, for example, discovering and accessing research assets or composing and executing application workflows. A typical VRE is often implemented as an integrated environment, including a catalog of research assets, a workflow management system, a data management framework, and tools for enabling user collaboration. In contrast, notebook environments like Jupyter allow researchers to rapidly prototype scientific code and share their experiments as online accessible notebooks. Jupyter can support several popular languages used by data scientists, such as Python, R, and Julia. However, such notebook environments do not have seamless support for running heavy computations on remote infrastructure or finding and accessing collaborative software code inside notebooks. This article investigates the gap between a notebook environment and a VRE and proposes an embedded VRE solution for the Jupyter environment called Notebook‐as‐a‐VRE (NaaVRE). The NaaVRE solution provides functional components via a component marketplace and allows users to create a customized VRE on top of the Jupyter environment. From the VRE, a user can search research assets (data, software, and algorithms), compose workflows, manage the lifecycle of an experiment, and share the results among users in the community. We demonstrate how such a solution can enhance a legacy workflow that uses Light Detection and Ranging (LiDAR) data from country‐wide airborne laser scanning surveys for deriving geospatial data products of ecosystem structure at high resolution over broad spatial extents. This enables users to scale out the processing of multi‐terabyte LiDAR point clouds for ecological applications to more data sources in a distributed cloud environment. Similar applications could be developed for workflows producing other essential biodiversity variables.

show abstract

Assessing and restoring reproducibility of Jupyter notebooks

Cited by 49 publications

References 36 publications

Ciência de Dados com Reprodutibilidade usando Jupyter

Ciência de Dados com Reprodutibilidade usando Jupyter

Restoring Execution Environments of Jupyter Notebooks

Notebook‐as‐a‐VRE (NaaVRE): From private notebooks to a collaborative cloud virtual research environment

Contact Info

Product

Resources

About