An integrated strategy for the analysis of student evaluation of teaching: from descriptive measures to explanatory models

Abstract

Over the last decade, the assessment of university teaching quality has assumed a prominent role in the university system with the main purpose of improving the quality of courses offered to students. As a result of this process, a host of studies on the evaluation of university teaching was devoted to the Italian system, covering different topics and considering case studies and methodological issues. Based upon this debate, the contribution aims to present an integrated strategy of analysis which combines both descriptive and model-based methods for the treatment of student evaluation of teaching data. More specifically, the joint use of item response theory and multilevel models allows, on the one hand, to compare courses’ ranking based on different indicators and, on the other hand, to define a model-based approach for building up indicators of overall students’ satisfaction, while adjusting for their characteristics and differences in the compositional variables across courses. The usefulness and the relative merits of the proposed procedure are discussed within a real data set

On the Use of Student Evaluation of Teaching: A Longitudinal Analysis Combining Measurement Issues and Implications of the Exercise

“…As said above, in the analysis of students' ratings of university teaching many external factors related to students, lecturers, courses, schedules or more generally, environmental characteristics or disturbances can affect the result of the evaluation exercise. Previous studies carried out on the topic (Rampichini et al 2004;La Rocca et al 2017) agree on indicating that the student characteristics (i.e., the personal and academic background or the student's self-assessment of her/his prior knowledge) are among the factors that account for the variability between ratings of QUT.…”

“…In that way we try to contribute on the debate on the adequacy of the unadjusted indicators of SET (based on point estimates and which ignore the effect of confounders) to provide reliable information on the QUT. Indeed, with reference to Italy, in the last decades, SET surveys earned a key role in assessing the overall performance of the university institutions and a number of point indicators based on SET data were developed and adopted at national and local level (La Rocca et al 2017).…”

Does the Type of Degree Predict Different Levels of Satisfaction and Loyalty? A Brand Equity Perspective

“…Despite the notable contributions of previous studies, research on branding evaluation of students' satisfaction in the context of disciplinary areas is limited. Some authors simply assessed whether level of student satisfaction varied across faculty (La Rocca et al, 2017) while others (Denson et al, 2010) compared student satisfaction across broad disciplines (natural and physical sciences vs. engineering and related technologies). No specific study has sought to examine how disciplinary differences can determine student evaluation of university brands.…”

The validity of student evaluation of teaching: Is there gender bias?

“…In other words, lecturers question students' competence to grade teachers, classes, and course syllabuses, and they also believe that the SET does not have one widely accepted definition of the concept "effective teaching" (Johnson, 2000). On the other hand, experts have tackled the SET on both national and international level (La Rocca, Parrella, Primerano, Sulis, & Vitale, 2017). The main concerns are related to the SET structure and questions, along with the validity of students' answers, which are susceptible to other factors rather than teaching (Maricic et al, 2016;Zhao & Gallant, 2012).…”