How Do Missing Data Bias Estimates of Within-Group Agreement? Sensitivity of SD WG, CVWG, rWG(J), rWG(J) * , and ICC to Systematic Nonresponse

Newman, Daniel A.; Sin, Hock Peng

doi:10.1177/1094428106298969

Cited by 67 publications

(84 citation statements)

References 46 publications

(95 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, we contend that clarifying the mixed results within the diversity literature requires attention to how, and from whom, researchers collect the data on which diversity variables are based (e.g., Riordan, 2001), how well researchers' conceptualizations of diversity match their operationalizations of diversity (e.g., Harrison & Klein, 2007), and how researchers handle the ubiquitous missing-data problem that hampers the accurate assessment of workgroup diversity (e.g., Allen, Stanley, Williams, & Ross, 2007b;Newman & Sin, 2009 Behav Res (2011) 43:508-521 DOI 10.3758/s13428-010-0053-9 specifically, we examine the effectiveness of rules that govern the decision to drop or retain groups based on within-group response rates. We do this in the context of diversity research, where diversity is operationalized using the standard deviation.…”

Section: Retention Rulesmentioning

confidence: 99%

Examining workgroup diversity effects: does playing by the (group-retention) rules help or hinder?

et al. 2011

View full text Add to dashboard Cite

Group diversity researchers are often faced with the problem of calculating diversity indices for groups that are incomplete due to participant nonresponse. Because participant nonresponse may attenuate the correlations that are observed between group diversity scores and outcome variables, some researchers use group-retention rules based on within-group response rates. With this approach, only those groups that have a within-group response rate at, or higher than, the rate prescribed by the group-retention rule are retained for subsequent analyses. We conducted two sets of experiments using computer simulations to determine the usefulness of group-retention rules. We found that groupretention rules are not a substitute for a high response rate and may decrease the accuracy of observed relations, and consequently, we advise against their use in diversity research.Keywords Missing data . Diversity . Groups . Retention rulesOver the past two decades, researchers and practitioners have shown considerable interest in understanding the impact that workgroup diversity has on group-level process variables (e.g., conflict, communication) and outcome variables (e.g., viability, innovation, performance). Some theoretical perspectives, such as the information-processing/decision-making approach (Ancona & Caldwell, 1992;De Dreu & West, 2001;Gruenfeld, Mannix, Williams, & Neale, 1996), predict that diversity will enhance processes and outcomes, whereas others, in particular the social identity, self-categorization, and similarity attraction perspectives (e.g., Byrne, 1971;Hogg & Abrams, 1988;Tajfel & Turner, 1986), predict that diversity will disrupt processes and impede outcomes. As various reviewers have concluded, the overall pattern of findings examining the effects of diversity is quite mixed (e.g., Horwitz & Horwitz, 2007;King, Hebl, & Beal, 2009;Mannix & Neale, 2005;Riordan, 2001;van Knippenberg, De Dreu, & Homan, 2004;Williams & O'Reilly, 1998). In their review, Mannix and Neale argued that these mixed results will only be clarified "by carefully considering moderators such as context, by broadening our view to include new types of diversity such as emotions and networks, and by focusing more carefully on mediating mechanisms" (p. 32). More recently, King et al. expressed a similar sentiment and emphasized the need, in future diversity research, to examine group processes and outcomes with greater specificity.We agree with these observations, but would add that methodological issues associated with diversity research also need careful scrutiny. Specifically, we contend that clarifying the mixed results within the diversity literature requires attention to how, and from whom, researchers collect the data on which diversity variables are based (e.g., Riordan, 2001), how well researchers' conceptualizations of diversity match their operationalizations of diversity (e.g., Harrison & Klein, 2007), and how researchers handle the ubiquitous missing-data problem that hampers the accurate assessment of workgroup dive...

show abstract

Section: Retention Rulesmentioning

confidence: 99%

Examining workgroup diversity effects: does playing by the (group-retention) rules help or hinder?

et al. 2011

View full text Add to dashboard Cite

show abstract

“…Using incomplete data to aggregate to the group-level will cause an overestimation of the group potency and agreement values [26]. Thus, to remove this bias, we corrected group potency values by using the Systematic Nonresponse Parameters (SNP) [27] approach for missing data.…”

Section: Methodsmentioning

confidence: 99%

Classification of Group Potency Levels of Software Development Student Teams

et al. 2015

View full text Add to dashboard Cite

Abstract-This paper describes the use of an automatic classifier to model group potency levels within software development projects. A set of machine learning experiments that looked at different group characteristics and various collaboration measures extracted from a team's communication activities were used to predict overall group potency levels. These textual communication exchanges were collected from three software development projects involving students living in the US, Turkey and Panama. Based on the group potency literature, group-level measures such as skill diversity, cohesion, and collaboration were developed and then collected for each team. A regression analysis was originally performed on the continuous group potency values to test the relationships between the group-level measures and group potency levels. This method, however, proved to be ineffective. As a result, the group potency values were converted into binary labels and the relationships between the group-level measures and group potency were re-analyzed using machine learning classifiers. Results of this new analysis indicated an improvement in the accuracy of the model. Thus, we were able to successfully characterize teams as having either low or high potency levels. Such information can prove useful to both managers and leaders of teams in any setting.

show abstract

“…The ICC coefficients assess group reliability. ICC(1) refers to the proportion of the total variance that can be explained by team membership (Newman & Sin, 2009). As stated by Dyer, Hanges, and Hall (2005) multilevel analyses may provide minimal practical benefits and may be difficult or impossible to estimate when ICC(1)s are smaller than .05.…”

Section: Analysesmentioning

confidence: 99%

“…ICC(1) values of the items ranged from .010 to .132 (M=.059), 12 items having an ICC(1) value below .05, and design effects were all below 2 (M=1.286, Min = 1.049, Max = 1.636). The ICC(2) coefficients indicate the reliability of the group means (Bartko, 1976;Newman & Sin, 2009). ICC(2) values of the items range from .13 to .53 (M=.35), indicating that aggregate group means are not reliable because of high levels of within-group variance.…”

Section: Analysesmentioning

confidence: 99%

Group, team, or something in between? Conceptualising and measuring team entitativity

Vangrieken

Boon

Dochy

et al. 2017

FLR

View full text Add to dashboard Cite

show abstract

How Do Missing Data Bias Estimates of Within-Group Agreement? Sensitivity of *SD _WG, CV_WG, r_WG(J), r_WG(J)** , and ICC to Systematic Nonresponse

Cited by 67 publications

References 46 publications

Examining workgroup diversity effects: does playing by the (group-retention) rules help or hinder?

Examining workgroup diversity effects: does playing by the (group-retention) rules help or hinder?

Classification of Group Potency Levels of Software Development Student Teams

Group, team, or something in between? Conceptualising and measuring team entitativity

Contact Info

Product

Resources

About