Psychometric properties of the Self‐assessment Practice Scale for professional training contexts: evidence from confirmatory factor analysis and Rasch analysis

Abstract

Self-assessment is a fundamental skill for professionals because self-assessment can promote self-regulated learning and professional development. However, studies reporting the use of self-assessment instruments in the professional training context are scarce. This study aimed to re-evaluate the psychometric properties of the Self-assessment Practice Scale (SaPS), which was originally developed in the school context, and extend its use to the professional training context. A sample of 200 investigative interviewer trainees from Australia and North America were invited to complete the modified SaPS. After removing misfitting items, the confirmatory factor analysis results confirmed a first-order four-factor solution. The multidimensional Rasch analysis demonstrated that the resultant 16 items had satisfactory fit to the Rasch model. In general, the results supported the use of the 16-item modified SaPS as a valid measure for the sample in this study. The potential of using the SaPS in professional training contexts is discussed.

Validation of a Subject-Specific Student Self-Assessment Practice Scale (SaPS) Among Secondary School Students in the Philippines

“…For the within-network validity, confirmatory factor analysis (CFA) and Rasch analysis were used. Numerous empirical studies have applied this complementary combined analytical approach (e.g., Chang & Engelhard, 2016; Testa et al, 2019; West et al, 2020; Yan, 2018a; Yan, Brubacher, Boud, Powell et al, 2020) to examine unique psychometric information of the scale items and subscales. The structural validity of the four-factor model of SaPS was assessed using CFA, where n = 19 participants with more than 5% of item-level missing data were excluded (Tabachnick, Fidell, & Ullman, 2007).…”

“…The SaPS and its short form and adapted versions have been used for graduate students (Yan, 2020b) and professional trainees in a Western cultural context (Yan, Brubacher, Boud, & Powell, 2020). The scale can track the development of self-assessment skills and can be used to inform teaching strategies that can promote self-assessment for optimizing learning outcomes.…”

“…Such a procedure has been used previously with closely related constructs (e.g., self-concept and self-efficacy; Lent, Brown & Gore, 1997). Contextualizing self-assessment practices has also been recommended in a separate validation study (Yan, Brubacher, Boud, & Powell, 2020). The engagement outcomes measured are also framed in the context of the English language subject.…”

“…Broadly speaking, self-assessment occurs when “students reflect on the quality of their work, judge the degree to which it reflects explicitly stated goals or criteria, and revise accordingly” (Andrade & Valtcheva, 2009, p. 13). Self-assessment is a core skill for self-regulated learning (Panadero & Alonso-Tapia, 2013; Yan, Brubacher, Boud, & Powell, 2020) and lifelong learning (Papanthymou, 2018; Siegesmund, 2017; Yan & Brown, 2017). It is linked with key learning outcomes, such as school achievement (McDonald & Boud, 2003; Yan, Chiu, & Ko, 2020), student motivation (McMillan & Hearn, 2008), and self-regulated learning (Panadero, Jonsson, & Botella, 2017), among others (see Andrade, 2019; Brown & Harris, 2013, for reviews).…”

“…Yan (2018a) adopted a theory-driven approach in developing the Self-assessment Practice Scale (SaPS) in the Hong Kong school context, which has been successfully applied in other contexts, such as higher education (Yan, 2020b) and professional training in western culture (Yan, Brubacher, Boud, & Powell, 2020). The instrument evaluates students’ use of two core self-assessment processes: seeking self-directed feedback and self-reflection.…”