No abstract

Over a period of 10 years, we have developed a sustainable process of online portfolio assessment that demonstrates both reliability and validity, using both qualitative and quantitative measures. The sustainable cycle is that, each semester, we assess a random sampling of the students' work that they have posted, as per our instructions, in an online portfolio. During the reading, the faculty score the documents for 11 variables, including writing, content, audience awareness, and document design. We achieved validity by a modified online Delphi that led to a redefinition of the construct of technical communication itself; we achieved reliability by adjudication resulting in adjacent scores. The results of our assessment meet the requirements of ABET and result in a continual cycle of improvement for our technical communication curriculum. Results from three semesters show an improving correlation between the course grade and the overall, holistic portfolio score.

This paper describes an innovative model for assessing the technical communication course by analytically scoring online portfolios, open to the internet, for ten separate (analytic) variables and one overall (holistic) score. The model is a statistically verifiable and sustainable method that strengthens the curriculum and fosters consensus within the teaching community. We achieved construct validity by redefining the elements of the course to incorporate communication in the digital age and then by creating new criteria for evaluation related to that construct. We achieved inter‐reader reliability by beginning each assessment with a calibrated reading and by adjudicating non‐adjacent scores. After using the model successfully for three semesters, we can see increased consistency in teaching among sections and semesters, more communication among instructors, and we are beginning a database with which we can test further change. The theory and method behind this model can be applied to other disciplines as well.

No abstract

No abstract