Variability across sources of performance ratings.

Fuqua, Dale R.; Johnson, Al W.; Newman, Jennifer; Anderson, Mark; Gade, Eldon M.

doi:10.1037/0022-0167.31.2.249

Cited by 18 publications

(9 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, while displaying caution and awareness of over-enthusiastic interpretations of results by authors, some predictions deriving from published quantita- tive peer assessment studies may be made. The following variables have been identified by researchers as mediating the correspondence between faculty and peer ratings: ability of student raters (Jacobs at al, 1975); practice effects (Orpen, 1982;Fuqua, Johnson, Newman, Anderson, & Gade, 1984;Hunter & Russ, 1996); number of student raters related to the reliability of marking (Magin, 1993); methodologies employed (Falchikov, 1986); the type of assessment involved (Mowl & Pain, 1995). In addition, Falchikov and Boud's (1989) meta-analysis identified the following significant variables in the context of student self assessment: the level of course; the complexity of measurements used; the explicitness of criteria and student ownership of these; the subject area in which the assessment takes place.…”

Section: Correlating Common Metrics With Context Variablesmentioning

confidence: 99%

Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks

Falchikov

Goldfinch

2000

Review of Educational Research

871

506

View full text Add to dashboard Cite

Forty-eight quantitative peer assessment studies comparing peer and teacher marks were subjected to meta-analysis. Peer assessments were found to resemble more closely teacher assessments when global judgements based on well understood criteria are used rather than when marking involves assessing several individual dimensions. Similarly, peer assessments better resemble faculty assessments when academic products and processes, rather than professional practice, are being rated. Studies with high design quality appear to be associated with more valid peer assessments than those which have poor experimental design. Hypotheses concerning the greater validity of peer assessments in advanced rather than beginner courses and in science and engineering rather than in other discipline areas were not supported. In addition, multiple ratings were not found to be better than ratings by singletons. The study pointed to differences between self and peer assessments, which are explored briefly. Results are discussed and fruitful areas for further research in peer assessment are suggested.

show abstract

Section: Correlating Common Metrics With Context Variablesmentioning

confidence: 99%

Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks

Falchikov

Goldfinch

2000

Review of Educational Research

871

506

View full text Add to dashboard Cite

show abstract

“…Israelite (1983): graduate students in an education course overrated themselves on three out of four assignments. Fuqua et al (1984): before training, the mean self rating of prepracticum counselor trainees was 18.61 (sd=2.14) compared with a mean of 17.25 (sd=4.06) for supervisors. After training, mean self rating was 21.33 (sd=2.43) and mean supervisor rating was 19.84 (sd=3.62).…”

Section: Evidence About Self-markingmentioning

confidence: 99%

Quantitative studies of student self-assessment in higher education: a critical analysis of findings

1989

View full text Add to dashboard Cite

Student self-assessment occurs when learners make judgements about aspects of their own performance. This paper focuses on one aspect of quantitative self-assessments: the comparison of student-generated marks with those generated by teachers. Studies including such comparisons in the context of higher education courses are reviewed and the following questions are addressed: (i) do students tend to over-or under-rate themselves vis-a-vis teachers?, (ii) do students of different abilities have the same tendencies?, (iii) do students in different kinds or levels of course tend to under-or over-rate themselves?, (iv) do students improve their ability to rate themselves over time or with practice?, (v) are the same tendencies evident when self-marks are used for formal assessment purposes?, and (vi) are there gender differences in self-rating? The paper also discusses methodological issues in studies of this type and makes recommendations concerning the analysis and presentation of information.

show abstract

“…The influence of extraneous factors specific to ratings of counseling performance has been documented from several different perspectives. For example, significant differences have been reported among ratings obtained from a counseling student, a peer group, and supervisors (Fuqua, Johnson, Newman, Anderson, & Gade, 1984). Different empirical structures have even been reported for ratings obtained from different sources (Jansen, Robb, 8c Bonk, 1973).…”

mentioning

confidence: 99%

The Effects of Order of Stimulus Presentation on Ratings of Counseling Performance

Newman

Fuqua

1986

Couns Edu & Supervision

Self Cite

View full text Add to dashboard Cite

This study was designed to examine the effects of order of stimulus presentation on observer ratings of counseling performance. Two videotaped counseling simulations, determined by judges to represent either high or low quality performance, served as the stimuli. Participants were randomly assigned to one of two levels of the order effect: Group 1 viewed the videotape with low quality performance first, followed by the high quality performance videotape; Group 2 viewed the high quality performance videotape first, followed by the low quality performance videotape. Four widely used measures of counseling performance served as dependent measures. Results revealed a statistically significant interaction between quality of performance and the order in which the performances were rated.The assessment of counseling performance plays an important role in both training and research. Assessment is central to the evaluation of student performance and progress. Accurate feedback, which is an essential element of effective training methods (Ford, 1979), is highly reliant on accurate assessment. The effectiveness of counselor training techniques is determined through assessment as well. One of the most widely used methods for assessing counseling performance is the rating scale (Tinsley & Weiss, 1975), which typically consists of items developed around specific characteristics of performance and presented on a multipoint scale. Observers are required to make subjective judgments regarding the degree to which each characteristic is present in a given Jody L.

show abstract

Variability across sources of performance ratings.

Cited by 18 publications

References 8 publications

Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks

Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks

Quantitative studies of student self-assessment in higher education: a critical analysis of findings

The Effects of Order of Stimulus Presentation on Ratings of Counseling Performance

Contact Info

Product

Resources

About