Analysis of Differences in Nonteaching Factors Influencing Student Evaluation of Teaching between Engineering and Business Classrooms

Abstract

In recent years, there have been increasing calls from the government and other organizations to provide easy public access to student evaluations of teaching. Indeed, the increasing ease of displaying and viewing large quantities of information, and competition among universities and majors for students, makes it likely that an era of greater transparency of this type of information is at hand. While students' evaluation of teaching (SET) is one quantitative metric that rates the instructor, it may be influenced by factors that are often beyond the instructor's control. In this study, we analyze a longitudinal data set from both engineering and business schools of a large public university, and identify factors that influence SET. We show which factors have the highest influence on overall SET scores, and contrast these between engineering and business colleges. Colleges within the same university may have differences in the factors affecting SET, and recognition of this is important in effectively and fairly evaluating SET scores. We also provide recommendations regarding information that should be displayed along with the SET, particularly when SET scores are made public, so that instructors are not unduly penalized when their evaluations can be influenced by factors over which they have no control.

Student Evaluations of Teaching, Course and Student Characteristics at Andrews University

“…They found that female students tended to give higher SET scores than male students (Ali & Ajmi, 2013;Al-Issa & Sulieman, 2007;Batten et al, 2013;Beuth et al, 2015;Driscoll & Cadden, 2010;Narayanan et al, 2014;Kozub, 2010;Shauki et al, 2009;Whitworth et al, 2002;Worthington, 2002).…”

“…They proposed that different non-instructional factors could affect the results of SET or produce biased results, and called for further studies (Coffman, 1954;Reynolds, 1979). Since then, different studies have been conducted to identify and understand the factors that could affect the results of SET (Abrami, Perry, & Leventhal, 1982;Ali, & Ajmi, 2013;Al-Issa & Sulieman, 2007;Centra, 1977;Frey, Leonard, & Beatty, 1975;Marsh, 1983;Narayanan, Sawaya, & Johnson, 2014;Santhanam & Hicks, 2002;Whitworth, Price, & Randall, 2002;Worthington, 2002). These studies uncovered different factors, which were not related to the dimensions that the SET is trying to measure, which influence the results of SET.…”

“…Also, the majority of the research that investigated student characteristics suggested that student status and gender affected the results of SET. These studies reported that gender, student status, course type, course level, and academic school influenced the SET, and this influence affected the reliability and validity of SET results (Ali & Ajmi, 2013;Narayanan, Sawaya, & Johnson, 2014).…”

“…Different studies indicate that SET scores are affected by some of the characteristics of the course and/or student (Al-Issa &Sulieman, 2007;Ali & Ajmi, 2013;Batten, Birch, Wright, Manley, & Smith, 2013;Beuth et al, 2015;Galbraith, Merrill, & Kline, 2012;Narayanan et al, 2014;Kozub, 2010;Pounder, 2007;Shauki, Ratnam, Fiedler, & Sawon, 2009;Whitworth et al, 2002;Worthington, 2002). These characteristics were: gender, student status, course type, course level, and academic school.…”

“…whether the courses and instructors were effective (Anantharaman, Lee, & Jones, 2010;Galbraith et al, 2012). Different researchers have recognized that need and searched for evidence that could help support the accuracy or inaccuracy of SET scores (Alauddin & Kifle, 2014;Ali & Al Ajmi, 2013;Choi & Kim, 2014;Korte, Lavin, & Davies, 2013;McCann & Gardner, 2013;Narayanan et al, 2014;Nargundkar & Shrikhande, 2012). The literature review indicated that some dimensions of effective teaching predicted SET overall rating, and suggested that gender, student academic status, course type, course level, and academic school were the student and course characteristics that affect SET score.…”