How do we rate? An evaluation of online student evaluations

Abstract

This paper analyses the popular RateMyProfessors (RMP) website where students evaluate instructors in higher education. A study was designed to measure (1) the awareness and utilisation of the RMP website, (2) the internal and external validity of the RMP ratings in measuring teaching effectiveness, and (3) variation in the above across disciplines. It is concluded that the category of ratings, created by the website, establishes an anti-intellectual tone that manifests itself in comments about instructors' personality, easiness of workload and entertainment value rather than knowledge attained. The Rate My Professors (RMP) website (ratemyprofessors.com) offers a community forum where 'students do the grading', anonymously sharing evaluations of instructors with other students. The website aims to make information available so students can choose better instructors and courses, thereby improving their education. The merits of the website are often contested within the walls of academe, especially by the faculty that are directly scrutinised, but largely this website lacks academic or public evaluation. Although the debate concerning the website usually centres on the quality of data provided by the website, there is little information to substantiate the credibility of the data. This paper contributes to the emerging literature (see Felton et al. 2004; Coladarci and Kornfield 2007) that provides information on the services and disservices of this popular website to the academic community. There are many similar websites, such as pickaprof.com, campusdirt.com, myprofessorsucks. com, and rateyourprof.com, but RMP is by far the most popular student rating site. Since RMP started in 1999,1 the for-profit website has observed great success and now sells ads to numerous international corporations such as VISA, Monster, MSN, Citibank and Allstate. In early 2004, RMP had almost 1.5 million ratings from nearly 4000 schools. Three years later, the number of postings increased by 300% to over six million. Currently over 6000 colleges and universities and nearly 800,000 instructors are listed across United States, Canada, England, Scotland and Wales. With over eight million student members, the daily traffic averages over 200,000 unique visitors per day. One thing is for sure, the website is not going away. Like it or not, the information on the website is very profitable.

Do Students Listen To Advice From Their Experienced Peers?

“…If a student is very pleased with the professor"s teaching or very angry at the professor, he or she is more likely to make a comment on RMP than someone who had an average experience (Davison, 2006).…”

On the Development and Application of a Framework for Understanding the Properties and Information Quality of Online Reputation Systems

“…For example, students can rate a professor more than one time, students can use false names, and professors can rate themselves. Additionally, the sample of students providing ratings may not be representative of the population of students enrolled in a course because students who have strong feelings about a course may be more likely to provide ratings than students without strong feelings about a course (Davison & Price, 2009;Johnson & Crews, 2013;Stonebraker & Stone, 2015). Silva et al (2008), however, note that there are more positive comments than negative comments posted on the Rate My Professors online reputation system which suggests that it is not only dissatisfied students who are motivated to spend time entering ratings into the system.…”

Normalisation of Technology Use in a Developing Country Higher Education Institution

“…Whilst student evaluations have largely been conducted physically using paper-based evaluation forms, many educational institutions are migrating to online evaluations [15], [38]. HEIs are leveraging on the advantages that an online evaluation could bring.…”