No abstract

Issues surrounding the psychometric properties of writing assessments have received ongoing attention. However, the reliability estimates of scores derived from various holistic and analytical scoring strategies reported in the literature have relied on classical test theory (CT), which accounts for only a single source of variance within a given analysis. Generalizability theory (GT) is a more powerful and flexible strategy that allows for the simultaneous estimation of multiple sources of error variance to estimate the reliability of test scores. Using GT, two studies were conducted to investigate the impact of the number of raters and the type of decision (relative vs. absolute) on the reliability of writing scores. The results of both studies indicated that the reliability coefficients for writing scores decline as (a) the number of raters is reduced and (b) when absolute decisions rather than relative decisions are made.Writing assessment is one of the most common types of performancebased testing. The importance of the decisions made using test results from performance-based assessments in applied settings makes it paramount that researchers establish the reliability of scores resulting from such assessments. Professionals charged with providing evidence for the psychometric properties of writing tests need to move beyond reliability estimates derived

No abstract

No abstract

The factorial structure of the performance ratings of 27 teaching practices contained on the North Carolina Teaching Performance Appraisal Instrument (TPAI) is reported in this paper. As currently used, ratings using the TPAI yield five scores: (a) Management of Instructional Time, (b) Management of Student Behavior, (c) Instructional Presentation, (d) Instructional Monitoring, and (e) Instructional Feedback. This five function scoring scheme is based on logical judgement and not empirical evidence. The results a study using factor analysis procedures suggest that a five factor solution that paralleled the current scoring scheme was not as parsimonious as a two factor solution. The clarity and meaningfulness of the interpretation of the two factor solution provides supportive evidence for the construct validity of the TPAI as well as suggestions for a more utilitarian procedure for using the instrument in large scale teaching performance assessment programs.