Why peer evaluations in student teams don’t tell us what we think they do

Mayfield, Clifton O.; Tombaugh, Jay R.

doi:10.1080/08832323.2018.1503584

Cited by 8 publications

(10 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we found moderate associations (average r = 0.35) between target effects and self-ratings, which is stronger than that reported in the previous studies of student teams (Loignon et al, 2017;Mayfield and Tombaugh, 2019). As with peer-peer consensus, peer-self agreement can be interpreted as evidence for the accuracy of both self-and peer-ratings (i.e., presumably the ratings of different raters converge because they are all influenced by a target's true skill level).…”

Section: Discussioncontrasting

confidence: 91%

“…Greguras et al (2001) found on average 26% of the variance in peer ratings was target variance and 29% was perceiver variance. Mayfield and Tombaugh (2019) found on average 22% of the variance in peer ratings was target variance and 36% was perceiver variance. Schmidt et al (2021) found on average 28% was target variance and 35% was perceiver variance.…”

Section: Discussionmentioning

confidence: 92%

“…In other words, do people who are rated by others as more/less competent also rate themselves as more/less competent? Whereas some research shows agreement between self-rated and peer-rated assessments (e.g., Watson and Clark, 1991), other research shows mixed results (e.g., Heintzelman et al, 2020, 13) or no significant agreement (Mayfield and Tombaugh, 2019). In the current data, significant positive correlations would suggest peer ratings and self-ratings both measure the same construct-presumably, teamwork skills; in contrast, weak or non-significant correlations would suggest that these ratings are a poor indicator of teamwork skills, which could be imputed to less competent individuals being particularly prone to overestimate their competence (Pennycook et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

“…A number of studies have explored how much peer or observer ratings reflect consensus about the target rather than rater biases or other influences (Kenny, 2020). However, to our knowledge, few previous peer-reviewed publications have used a social relations model to assess consensus in the peer-ratings made by members of student teams (e.g., Greguras et al, 2001;Mayfield and Tombaugh, 2019;Schmidt et al, 2021). Therefore, our first goal was to add to this limited literature by assessing the consensus in peer-ratings using a different sample, setting, and rating system.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

What Do Peer Evaluations Represent? A Study of Rater Consensus and Target Personality

Martin

Locke

2022

Front. Educ.

View full text Add to dashboard Cite

When professors assign group work, they assume that peer ratings are a valid source of information, but few studies have evaluated rater consensus in such ratings. We analyzed peer ratings from project teams in a second-year university course to examine consensus. Our first goal was to examine whether members of a team generally agreed on the competence of each team member. Our second goal was to test if a target’s personality traits predicted how well they were rated. Our third goal was to evaluate whether the self-rating of each student correlated with their peer rating. Data were analyzed from 130 students distributed across 21 teams (mean team size = 6.2). The sample was diverse in gender and ethnicity. Social relations model analyses showed that on average 32% of variance in peer-ratings was due to “consensus,” meaning some targets consistently received higher skill ratings than other targets did. Another 20% of the variance was due to “assimilation,” meaning some raters consistently gave higher ratings than other raters did. Thus, peer ratings reflected consensus (target effects), but also assimilation (rater effects) and noise. Among the six HEXACO traits that we examined, only conscientiousness predicted higher peer ratings, suggesting it may be beneficial to assign one highly conscientious person to every team. Lastly, there was an average correlation of.35 between target effects and self-ratings, indicating moderate self-other agreement, which suggests that students were only weakly biased in their self-ratings.

show abstract

Section: Discussioncontrasting

confidence: 91%

Section: Discussionmentioning

confidence: 92%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

What Do Peer Evaluations Represent? A Study of Rater Consensus and Target Personality

Martin

Locke

2022

Front. Educ.

View full text Add to dashboard Cite

show abstract

“…A conscious decision was made to exclude peer evaluations from the list, as this would lead to the undesirable situation of two students de facto grading each other—a scenario that is especially unattractive since research on peer evaluations indicates that measuring performance in this way can be quite imprecise (Mayfield & Tombaugh, 2019). Note that in all included methods, the ultimate decision of any consequence for a suspected free-rider remained with the course coordinator (Cook, 1981).…”

Section: Methodsmentioning

confidence: 99%

A Students’ Preferences-Based Approach to Select Methods for Detecting and Handling Free-Riding

Herik

Benning

2021

Journal of Marketing Education

View full text Add to dashboard Cite

Free-riding is a serious challenge in group projects. While there are various methods to reduce free-riding, marketing educators still face a difficult task when selecting an appropriate method for their course. In this study, we propose a students’ preferences-based approach that supports marketing educators with the selection of methods to detect and handle free-riding. To measure these preferences, students completed an online survey based on a choice task about two methods to detect free-riding and a ranking task about four methods to handle free-riding ( n = 254). Their answers were analyzed using chi-squared tests, Borda scores, and rank-ordered logit models. The results show that (a) neither Dutch nor international students have a clear preference for one of the two detection methods (the reporting system vs. the process evaluation system), (b) grade discussion (a possible reduction of the free-rider’s grade based on a conversation with the course coordinator about each student’s contribution) is the most preferred method to handle free-riding, and (c) international students have a stronger preference for stricter handling methods. Marketing educators can apply the proposed approach, or use our specific findings, for designing methods to reduce free-riding in their courses.

show abstract

An exploration of psychological safety and conflict in first‐year engineering student teams

Huerta,

Sajadi,

Schibelius

et al. 2024

J of Engineering Edu

View full text Add to dashboard Cite

BackgroundDeveloping teamwork skills is a central objective of engineering education. Psychological safety and conflict management are pivotal components of teamwork, yet despite their significance, research in engineering project‐based learning (PBL) contexts is scant. Understanding students' experiences with psychological safety and its interaction with conflict is crucial to inform PBL pedagogy.PurposeThis study delves into first‐year engineering students' experiences of psychological safety and conflict, including their evolution in a PBL course.MethodsThroughout the semester, we collected data from 82 students via written reflections and focus groups. Employing a thematic analysis underpinned by the team conflict dynamics model, we characterized students' experiences with psychological safety and conflict.FindingsAt the semester's outset, psychological safety was notably lacking. Students often hesitated to share ideas due to apprehensions about peer reactions and fears of negative judgments. As the semester advanced, consistent positive affirmations nurtured psychological safety, increasing students' confidence and readiness to discuss ideas openly and engage in healthy task conflict. Notably, process conflicts arising from absenteeism, poor communication, and procrastination were prevalent across teams. When unresolved, these conflicts eroded psychological safety, intensifying stress, exacerbating frustrations, and provoking relationship conflict.ConclusionsOur study underscores the intertwined nature of psychological safety and conflict in shaping the first‐year design experience in student teams. We urge faculty to recognize their pivotal role in fostering an inclusive culture and highlight pedagogical strategies that can bolster psychological safety at the onset, encourage healthy task conflict, and monitor unhealthy process and relationship conflicts.

show abstract

Why peer evaluations in student teams don’t tell us what we think they do

Cited by 8 publications

References 66 publications

What Do Peer Evaluations Represent? A Study of Rater Consensus and Target Personality

What Do Peer Evaluations Represent? A Study of Rater Consensus and Target Personality

A Students’ Preferences-Based Approach to Select Methods for Detecting and Handling Free-Riding

An exploration of psychological safety and conflict in first‐year engineering student teams

Contact Info

Product

Resources

About