Measurement Invariance of Student Evaluation of Teaching across Groups defined by Course-Related Variables

Abstract

In the present study, comparability of scores from student evaluation of teaching forms was investigated. This is an important issue because scores given by students are used in decision making in higher education institutions. Three course-related variables (grade level, course type, and course credit) were used to define student subgroups. Then, multi-group confirmatory factor analysis was used to assess invariance of factorial structure, factor loadings and factor means across groups. It was found that although a common factorial structure held across groups, fully invariant factor loadings were observed only across instructors who teach different course types. For other groups, only partial invariance of factor loadings was obtained. Analyses also revealed that none of the subgroups had invariant factor means, indicating a possible bias. Results indicate that comparison of instructors based on student ratings may not be valid as it is mostly assumed.

Criteria for Teaching Performance in Psychology: Invariance According to Age, Sex, and Academic Stage of Peruvian Students

“…Furthermore, the education level or academic stage (current term attended by the undergraduate student) showed partial invariance evidence (only in the weak and strong models). The academic stage level (educational level) of the students has been referred to as an important factor to assess the measurement invariance of the didactic performance of professors, according to the assessment from students (Marsh, 1984;Marsh and Hocevar, 1984;Kalender, 2015).…”

“…Other aspects have been included for these analyses, such as: Validity of measurement constructs (Hornstein, 2017), the professor's and the students' sex (Boring, 2015;Eouanzoui and Jones, 2017), their semester or academic stage and type of class (Kalender, 2015), a same class taught at different academic semesters or years (Marsh and Hocevar, 1984), the grades obtained by students and the professor's qualification (Üstünlüoglu and Güngör-Culha, 2012;Spooren et al, 2017), or the same degree program or discipline taught at different institutions (Müller et al, 2017).…”

“…Factorial invariance was calculated including three variables that might indicate differences in the teaching performance assessment from students in didactic interactions: age, sex, and academic stage of the students. These variables have been reported in the current literature as control sources of factorial invariance of the instruments to identify student and professor performance assessment (Kalender, 2015;Kalender and Berberoglu, 2019). A multigroup confirmatory factor analysis (MGCFA) procedure was followed, closely related to the analysis models proposed by Byrne (2008) and Hirschfeld and Von Brachel (2014).…”

“…A second hypothesis was the existence of significant differences in the evaluation of the didactic performance of the teacher, according to sex, age groups, and academic level (stage) of the psychology students. To date, little is known about possible differences according to the age of the student body in the appraisal of teaching in the context of University education, but differences in students' appraisal of teaching according to their level of advancement in their studies have been reported (Marsh and Hocevar, 1984;Kamran et al, 2012;Kalender, 2015;Müller et al, 2017;Kitto et al, 2019;Mocanu et al, 2021;Pérez-Villalobos et al, 2021). Similarly, differences have been reported according to the gender of the student body, with respect to the students' assessment of teaching and the performance of their teachers, in the context of higher education (Boring, 2015;Boring et al, 2016;Potvin and Hazari, 2016;Eouanzoui and Jones, 2017;Heffernan, 2021;Kreitzer and Sweet-Cushman, 2021;Valencia, 2021).…”

The Examination of Measurement Invariance of Motivations to Become a Teacher Scale in Terms of Department and Gender

“…This concept is defined as measurement invariance and is needed to be able to compare these groups meaningfully [25] but usually is not tested [26]. The number of studies on the measurement invariance of instruments are scarce [27] and no studies were encountered yet on the MI of scales measuring motivations to become a teacher. At this point, TEDS-M Intentions/Motivation to be a teacher‖ scale [21] which was adapted from English into Turkish and shown to be valid and reliable (total variance explained by the whole scale = % 50, alpha = 0.75 Mc Donald Omega = 0.85, item-total correlations above 0.54) is an important and robust measurement instrument [28].…”