Validity and Reliability of Portfolio Assessment of Student Competence in Two Dental School Populations: A Four‐Year Study

Abstract

The purpose of this study was to empirically investigate the validity and reliability of portfolio assessment in two U.S. dental schools using a uniied framework for validity. In the process of validation, it is not the test that is validated but rather the claims (interpretations and uses) about test scores that are validated. Kane's argument-based validation framework provided the structure for reporting results where validity claims are followed by evidence to support the argument. This multivariate generalizability theory study found that the greatest source of variance was attributable to faculty raters, suggesting that portfolio assessment would beneit from two raters' evaluating each portfolio independently. The results are generally supportive of holistic scoring, but analytical scoring deserves further research. Correlational analyses between student portfolios and traditional measures of student competence and readiness for licensure resulted in signiicant correlations between portfolios and National Board Dental Examination Part I (r=0.323, p<0.01) and Part II scores (r=0.268, p<0.05) and small and non-signiicant correlations with grade point average and scores on the Western Regional Examining Board (WREB) exam. It is incumbent upon the users of portfolio assessment to determine if the claims and evidence arguments set forth in this study support the proposed claims for and decisions about portfolio assessment in their respective institutions.

Validity and reliability of a self‐assessment scale for Dental and Oral Health student's perception of transferable skills in Australia

“…The transferable skills survey, in particular the four‐factor structure for future transferable skills, also had good discriminant validity to differentiate the students' perception of transferable skills across different dental programmes. Valid instruments with proven reliability and capable of consistent measurements are becoming increasingly important for dental educators who seek to understand their students' perception of transferable skills …”

Validity and User Experience in an Augmented Reality Virtual Tooth Identification Test

“…Validity refers to accumulation of the gathered evidence from multiple sources to support the claim that the assessment is indeed evaluating the speciic knowledge and skills being tested. 24 When a new method of testing is contemplated, a validity study to examine format, content, context efects, and ease of use should be conducted prior to implementation. 13 The results of this study found moderate correlation of the AR virtual tooth identiication test and the traditional real tooth identiication test with other criterion measures used to determine student performance in the dental anatomy course.…”

Dental Students’ Clinical Experience Across Three Successive Curricula at One U.S. Dental School

“…Like many U.S. dental schools over the last 25 years, the University of California, San Francisco (UCSF) School of Dentistry has employed various clinical curricula to ensure students obtain adequate clinical experience and attain required competencies for graduation. Dental schools have introduced such innovative curriculum and assessment formats as comprehensive care, 1 – 6 case completion, 7 , 8 competency examinations, 9 – 12 portfolios, 13 , 14 and community‐based, off‐campus externships 15 – 17 . UCSF has had three major clinical curricular changes since 1992, moving from a discipline‐based curriculum to a comprehensive care curriculum to a procedural requirement curriculum plus externships.…”