Profiling Instructional Effectiveness to Reveal Its Relationship to Learning

Abstract

The purpose of the present study is to define instructional profiles and investigate the relationship between these profiles and learning indicators such as endof-semester grades and self-reported amount of learning. Instructional profiles were obtained using a segmentation method. Student ratings were used as indicators of instructional effectiveness. Results revealed that instructors who receive higher scores from students seem to be effective instructors in learning. However, instructors with high ratings from students did not receive high scores for all measures of instructional effectiveness. Effective instructors seem to have varying scores due to the imperfect relationship between instructional effectiveness and learning. It can be concluded that the definition of an effective instructor can vary across subgroups. For an instructor to be defined as effective, it is not necessary for them to receive higher scores for all measures. Low-rated aspects of effectiveness can be compensated for by showing high performance in other areas. Based on the results of the present study, instructional profiles or any other related traits should be investigated under subgroups that show differences.

Measurement Invariance of Student Evaluation of Teaching across Groups defined by Course-Related Variables

“…The main reason for this is because what SET forms measure, i.e., instructional effectiveness, is a multifaceted trait. Kalender (2014) found that different groups have varying definitions of instructional effectiveness based on SET forms. A similar study by Young and Shaw (1999) also defined different instructional profiles based on students' responses to SET forms.…”

“…Additional support came from studies by Trivedi, Pardos and Heffernan (2011) and Marsh and Hocevar (1991) who identified several subgroups of students showing distinct characteristics different than whole body. Furthermore, it was shown that subgroups students might exhibit varying behaviours in SET forms (Kalender, 2014;Smith & Cranton, 1992;Tenenbaum, 1977). It was the existence of such groups with varying responses to SET items that motivated the researcher to study on invariance of SET forms.…”

Reliability-Related Issues in the Context of Student Evaluations of Teaching in Higher Education

“…On the other hand, another cluster which included classes with low reliabilities was shown to have strong construct validity. Indeed unfitting models were expected since, as Young and Shaw (1999) and Kalender (2014), showed that there is no universal definition of instructional effectiveness acceptable for all. Thus it can be said that when confronted with low reliabilities, checking construct validity should be step which is needed to be checked.…”