A mean for all seasons

Weiss, David J.; Edwards, Ward

doi:10.3758/bf03192740

Cited by 4 publications

(2 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Statistical processing of CWS ratios was carried out using square roots, as recommended byWeiss and Edwards (2005). The reported mean CWS is the square of the sum of the individual square roots.…”

mentioning

confidence: 99%

People who judge people

Weiss

Shanteau

Harries

2006

Behavioral Decision Making

Self Cite

View full text Add to dashboard Cite

Experts who judge people usually provide opinions. It can be challenging to evaluate the professional performance of those experts, because for many domains there is no applicable external standard against which to verify the opinions. We review traditional methods for assessment and propose the purely empirical CWS approach as an alternative. Expert judgment entails discriminating among the various stimuli within the domain as well as being consistent when judging similar stimuli. We combine observed measures of these two components to form a ratio that we call the CWS index of expertise. We demonstrate the value of the index in an analysis of prioritization judgments made by occupational therapy students before and after they received specific training. The students' CWS scores improved considerably after training. The promise of the index as a selection tool is supported by the positive correlation of pre-training scores with both post-training scores and with course grades.

show abstract

mentioning

confidence: 99%

People who judge people

Weiss

Shanteau

Harries

2006

Behavioral Decision Making

Self Cite

View full text Add to dashboard Cite

show abstract

“…The results showed a range of scores between 0.72 and 10.14 (mean of all participants' scores was 2.56). To calculate mean, Weiss and Edward (45) provided an equation to average CWS scores when stimuli are the same for participants. In this procedure, first the square root of each CWS is taken, and then their mean is computed.…”

Section: Studymentioning

confidence: 99%

Is length of experience an appropriate criterion to identify level of expertise?

Rassafiani¹

2009

Scandinavian Journal of Occupational Therapy

View full text Add to dashboard Cite

Clinical experience plays an important role in the development of expertise, particularly when coupled with reflection on practice. There is debate, however, regarding the amount of clinical experience that is required to become an expert. Various lengths of practice have been suggested as suitable for determining expertise, ranging from five years to 15 years. This study aimed to investigate the association between length of experience and therapists' level of expertise in the field of cerebral palsy with upper limb hypertonicity using an empirical procedure named Cochrane-Weiss-Shanteau (CWS). The methodology involved re-analysis of quantitative data collected in two previous studies. In Study 1, 18 experienced occupational therapists made hypothetical clinical decisions related to 110 case vignettes, while in Study 2, 29 therapists considered 60 case vignettes drawn randomly from those used in Study 1. A CWS index was calculated for each participant's case decisions. Then, in each study, Spearman's rho was calculated to identify the correlations between the duration of experience and level of expertise. There was no significant association between these two variables in both studies. These analyses corroborated previous findings of no association between length of experience and judgemental performance. Therefore, length of experience may not be an appropriate criterion for determining level of expertise in relation to cerebral palsy practice.

show abstract

Criteria for performance evaluation

et al. 2009

View full text Add to dashboard Cite

Using a cognitive task (mental calculation) and a perceptual-motor task (stylized golf putting), we examined differential proficiency using the CWS index and several other quantitative measures of performance. The CWS index (Weiss & Shanteau, 2003) is a coherence criterion that looks only at internal properties of the data without incorporating an external standard. In Experiment 1, college students (n = 20) carried out 2- and 3-digit addition and multiplication problems under time pressure. In Experiment 2, experienced golfers (n = 12), also college students, putted toward a target from nine different locations. Within each experiment, we analyzed the same responses using different methods. For the arithmetic tasks, accuracy information (mean absolute deviation from the correct answer, MAD) using a coherence criterion was available; for golf, accuracy information using a correspondence criterion (mean deviation from the target, also MAD) was available. We ranked the performances of the participants according to each measure, then compared the orders using Spearman’s rs. For mental calculation, the CWS order correlated moderately (rs =.46) with that of MAD. However, a different coherence criterion, degree of model fit, did not correlate with either CWS or accuracy. For putting, the ranking generated by CWS correlated .68 with that generated by MAD. Consensual answers were also available for both experiments, and the rankings they generated correlated highly with those of MAD. The coherence vs. correspondence distinction did not map well onto criteria for performance evaluation.

show abstract

A mean for all seasons

Cited by 4 publications

References 40 publications

People who judge people

People who judge people

Is length of experience an appropriate criterion to identify level of expertise?

Criteria for performance evaluation

Contact Info

Product

Resources

About