An analysis of design process and performance in distributed data science teams

Abstract

PurposeOften, it is assumed that teams are better at solving problems than individuals working independently. However, recent work in engineering, design and psychology contradicts this assumption. This study aims to examine the behavior of teams engaged in data science competitions. Crowdsourced competitions have seen increased use for software development and data science, and platforms often encourage teamwork between participants.Design/methodology/approachWe specifically examine the teams participating in data science competitions hosted by Kaggle. We analyze the data provided by Kaggle to compare the effect of team size and interaction frequency on team performance. We also contextualize these results through a semantic analysis.FindingsThis work demonstrates that groups of individuals working independently may outperform interacting teams on average, but that small, interacting teams are more likely to win competitions. The semantic analysis revealed differences in forum participation, verb usage and pronoun usage when comparing top- and bottom-performing teams.Research limitations/implicationsThese results reveal a perplexing tension that must be explored further: true teams may experience better performance with higher cohesion, but nominal teams may perform even better on average with essentially no cohesion. Limitations of this research include not factoring in team member experience level and reliance on extant data.Originality/valueThese results are potentially of use to designers of crowdsourced data science competitions as well as managers and contributors to distributed software development projects.

A Multidimensional Framework to Uncover Insights of Group Performance and Outcomes in Competitive Environments With a Case Study of FIFA World Cups

“…However, the interactions between competitors may not be direct or evident in all contexts, and their meaning can vary from what we observed in our case study. For example, in soccer, the interactions with competitors are the attacks and are essential to winning a match; however, in competitive crowdsourcing, these interactions are not direct or clear, and these interactions may be related to sharing information [43,53]. In politics, the interactions among different political parties with different objectives may reveal temporal alliances [54].…”

“…The study of group performance in competitive environments has different perspectives and purposes: 1) improving teams' performance and making informed decisions by studying physical and technical factors of players, teams, and opponents [30,40,41], 2) test the accuracy of outcome prediction when developing machine learning algorithms, and statistical methods [42], 3) from a design perspective identify how groups' size, collaboration, and composition, among many other factors, impact performance [43].…”

Facilitating team-based data science: Lessons learned from the DSC-WAV project

“…Data science continues to grow within both academia [2,8,13,19,35] and industry [5,28]. In industry, where the kind of multifaceted problems that companies face require input from groups of people with diverse sets of skills, experiences, and backgrounds, team-based data science is the norm [33,53]. Moreover, the data science staff of many companies now work largely or exclusively remotely-a shift that began before the pandemic but accelerated greatly during it [30,42].…”

Facilitating team-based data science: lessons learned from the DSC-WAV project

“…Data science continues to grow within both academia [2,8,13,22,38] and industry [5,30]. In industry, where the kind of multifaceted problems that companies face require input from groups of people with diverse sets of skills, experiences, and backgrounds, team-based data science is the norm [35,57]. Moreover, the data science staff of many companies now work largely or exclusively remotely-a shift that began before the pandemic but accelerated greatly during it [32,45].…”

“…This work has spanned factors such as team size [1,48], composition [36], leadership [6], and autonomy [31,51]. The empirical results from this corpus of work are varied and have led some scholars to suggest that optimal conditions for these factors may be task-dependent [35,37].…”