Factors that Influence the Selection of a Data Science Process Management Methodology: An Exploratory Study

Abstract

This paper explores the factors that impact the adoption of a process methodology for managing and coordinating data science projects. Specifically, by conducting semi-structured interviews from data scientists and managers across 14 organizations, eight factors were identified that influence the adoption of a data science project management methodology. Two were technical factors (Exploratory Data Analysis, Data Collection and Cleaning). Three were organizational factors (Receptiveness to Methodology, Team Size, Knowledge and Experience), and three were environmental factors (Business Requirements Clarity, Documentation Requirements, Release Cadence Expectations). The research presented in this paper extends recognized factors for IT process adoption by bringing together influential factors that apply to data science. Teams can use the developed process adoption model to make a more informed decision when selecting their data science project management process methodology.

Current approaches for executing big data science projects—a systematic literature review

“…Process adoption papers discussed the key factors as well as the challenges for a data science team to adopt a new process. Specifically, the papers that discussed process adoption considered questions such as acceptance factors ( Saltz, 2017 , 2018 ; Saltz & Hotz, 2021 ), project success factors ( Soukaina et al, 2019 ), exploring the application of software engineering practices in the data science context ( Saltz & Shamshurin, 2017 ), and would deep learning impact a data science teams process adoption ( Shamshurin & Saltz, 2019a ).…”

Data Science Methodologies: Current Challenges and Future Approaches

“…Considering a survey from KDnuggets in 2014 [10], the main methodology used by 43% of responders was CRISP-DM. This methodology has been consistently the most commonly used for analytics, data mining and data science projects, for each KDnuggets poll starting in 2002 up through the most recent 2014 poll [11]. Despite its popularity, CRISP-DM was created back in the mid-1990 and has not been revised since its creation.…”