Assessing proxies of knowledge and difficulty with rubric‐based instruments

Abstract

The fields of psychometrics, economic education, and education have developed statistically‐valid methods of assessing knowledge and learning. These methods include item response theory, value‐added learning models, and disaggregated learning. These methods, however, focus on multiple‐choice or single response assessments. Faculty and administrators routinely assess knowledge through papers, thesis presentations, or other demonstrations of knowledge assessed with rubric rows. This paper presents a statistical approach to estimating a proxy for student ability and rubric row difficulty. Moreover, we have developed software so that practitioners can more easily apply this method to their instruments. This approach can be used in researching education treatment effects, practitioners measuring learning outcomes in their own classrooms, or estimating knowledge for administrative assessment. As an example, we have applied these new methods to projects in a large Labor Economics course at a public university.