How Data Scientists Use Computational Notebooks for Real-Time Collaboration

Abstract

Effective collaboration in data science can leverage domain expertise from each team member and thus improve the quality and efficiency of the work. Computational notebooks give data scientists a convenient interactive solution for sharing and keeping track of the data exploration process through a combination of code, narrative text, visualizations, and other rich media. In this paper, we report how synchronous editing in computational notebooks changes the way data scientists work together compared to working on individual notebooks. We first conducted a formative survey with 195 data scientists to understand their past experience with collaboration in the context of data science. Next, we carried out an observational study of 24 data scientists working in pairs remotely to solve a typical data science predictive modeling problem, working on either notebooks supported by synchronous groupware or individual notebooks in a collaborative setting. The study showed that working on the synchronous notebooks improves collaboration by creating a shared context, encouraging more exploration, and reducing communication costs. However, the current synchronous editing features may lead to unbalanced participation and activity interference without strategic coordination. The synchronous notebooks may also amplify the tension between quick exploration and clear explanations. Building on these findings, we propose several design implications aimed at better supporting collaborative editing in computational notebooks, and thus improving efficiency in teamwork among data scientists.

Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models

“…Computational notebooks are positioned as a potential solution to both support collaborative coding and communicating results to stakeholders [78]. However, a recent study reported reluctance for data scientists to directly communicate the in-progress model work in notebooks [65].…”

“…For P2, domain experts gave an overview and touched on the basic concepts of each class. P3 pair-authored [78] with domain experts to bridge concepts and a mathematical formula that encapsulates the information. With this iterative learning process, data scientists were able to kick start model building.…”

B2

“…These jumps cause changes in context, both in terms of the program state and analysts' mental models. The challenge of managing segments of analysis state is also faced in collaboration settings, where analysts sometimes jump through cells and need to understand cell dependencies [55]. Supporting analysts in navigating between segments of analysis in space and time poses additional challenges for the layout and temporal gaps.…”

AutoAIViz

“…The challenge is that this process is labor intensive, requiring input from multiple specialists with different skill sets [1,31,34,47,48]. As a result, AI and Human-Computer Interaction (HCI) researchers have investigated how to design systems with features that support data scientists in creating machine learning models [24,28,34,41,42,44]. This Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”

“…Kross & Guo users expressed a strong desire for an integrated user interface with both code and narrative. These needs are perhaps best captured in the narrative uses [24] of the Jupyter Notebook environment [20,21], and researchers have conducted numerous studies of how data scientists incorporate notebooks into their workflows [38,41], how they conduct version control for notebooks [23], and how they enable simultaneous multi-user editing in notebooks [42].…”