Validity Evidence of an Electronic Portfolio for Preservice Teachers

Abstract

This study applied Messick's unified, multifaceted concept of construct validity to an electronic portfolio system used in a teacher education program. The subjects included 128 preservice teachers who recently completed their final portfolio reviews and student teaching experiences. Four of Messick's six facets of validity were investigated for the portfolio in this study, along with a discussion of the remaining facets examined in two previous studies. The evidence provided support for the substantive and generalizability aspects of validity, and limited support for the content, structural, external, and consequential aspects of validity. It was suggested that the electronic portfolio may be used as one requirement for certification purposes, but may not be valid for the purpose of assessing teacher competencies.

Is the edTPA the Right Choice for Evaluating Teacher Readiness?

“…Again, it is concerning that only 10 percent of all edTPA submissions are randomly selected to be scored by a second independent scorer (SCALE 2013, 23). While there have been issues reported with interrater reliability in scoring regular holistic TPP portfolios (e.g., Carpenter, Marissa, and Bloom 1995;Derham and Diperna 2007;Koretz 1998;Oskay, Schallies, and Morgil 2008;Yao et al 2008), there is a strong body of research illustrating how these issues can be rectified (e.g., Baume and York 2002;Draves 2009;Johnson, McDaniel, and Willeke 2000;Meeus, Van Petegem, and Engels 2009;van der Schaaf, Stokking, and Verloop 2005). Recent research on music education TPPs illustrates that the content-specific pedagogical knowledge of the raters is critical when evaluating electronic portfolios for the connections between the artifacts within portfolio (Parkes 2013).…”

Interrater Reliability Estimators Commonly Used in Scoring Language Assessments: A Monte Carlo Investigation of Estimator Accuracy

“…Generalizability coefficients range from 0 to 1, where values closer to 1 indicate higher levels of reliability. The generalizability/dependability coefficient has been reported in studies related to performance assessments (R. L. Johnson, McDaniel, & Willeke, 2000;Nie, Yeo, & Lau, 2007;Yao et al, 2008).…”

Implementing Electronic Portfolios Through Social Media Platforms: Steps and Student Perceptions

“…One of the factors influencing the quality of portfolio entries is prompting [35]. Initially, when students are asked to reflect on their learning or experiences, they tend to use description, opinion, and personal anecdote [16] [36]. Although these writing formats are useful, they do not enable the same level of reflection and self-assessment as characterized by analytical or expository writing [11][12].…”

“…Converting artifacts to portable document format (PDF) or screenshot images simplifies access for viewers. Similar to previous methods, structured questions or prompts facilitate quality writing for this type of entry, such as 1) description of the artifact, 2) description of how the artifact shows evidence of meeting related professional standards, 3) assessment of competence gained from the experience, 4) analysis of impact on professional effectiveness, and 5) prediction of future training for improvement [36]. A final method, descriptively named blog buddies, engages students through a system of peer assessment.…”

“…[36].Indeed, Shepherd & Hannafin indicate that effective portfolio use involves multiform supports, such as tutorials, prompts, and even mentoring [42].…”