Lexical based automated teaching evaluation via students’ short reviews

Abstract

Student evaluations of teaching (SET) have become a popular approach to assess faculties’ teaching. Question‐score‐based questionnaire is the most common SET measure adopted in universities. However, it fails to cover important facets of teaching process that not mentioned in the predefined questionnaire, which can be substantially obtained from students’ short reviews. In this paper, we propose two lexical‐based methods, specifically knowledge‐based and machine learning‐based, to automatically extract opinions from short reviews. Furthermore, the diversity of reviews’ themes and styles of same sentiment polarity reviews can be observed from the extracted opinion results. The experimental results show that the proposed methods are able to achieve accuracies of 78.13 and 84.78%, respectively in the task of student review sentiment classification. Further investigation on linguistic features shows that reviews with same sentiment polarity shares similar language patterns. Finally, we present an application scenario in real SET process by utilizing aforementioned methods and discoveries.

High-quality vs low-quality teaching

“…Although the study of student teaching evaluations has retained its popularity, much of prior scholarly effort has been devoted to understanding effective designs and delivery mechanisms of student teaching evaluations in higher education institutions (Bi, 2018; Chulkov and Van Alstine, 2012; Gamliel and Davidovitz, 2005; Lin et al , 2019; Marsh, 1991). It is noteworthy that existing scholarly effort has predominantly focused on analyzing the pre-defined criteria developed by faculty and administrators in higher education institutions (Lin et al , 2019; Subbaye and Vithal, 2017). Consequently, there has been a lack of understanding of students’ sentiments regarding effective and ineffective teaching encompasses.…”

“…First, perhaps, because of the need for standardization and measurement efficiency, the vast majority of the formal institutional SET often contains criteria such as “the instructor clearly defined and explained the course objectives and expectations,” “the instructor showed a genuine interest in teaching the course” and “the instructor gave adequate instructions concerning assignments.” Because these formal institutional SET criteria are often defined by faculty and administrators, understanding teaching quality using this approach is, consequently, bounded by what is perceived as quality teaching from the perspective of faculty and administrators (Subbaye and Vithal, 2017). Furthermore, the existing institutional SET seems to be uni-dimensional where higher scores on the pre-defined teaching criteria represent high-quality teaching and lower scores suggest low-quality teaching (Lin et al , 2019). This particular approach may then fall short of capturing the breadth and sphere of elements of teaching effectiveness.…”

Sentiment analysis on massive open online course evaluations: A text mining and deep learning approach

“…In the presented scheme, a word with semantic and contextual information in short-and longdistance periods has been represented. Lin et al [40] examined the predictive performance of knowledge-based and machine learning-based approaches for sentiment analysis on student evaluations of teaching. In another study, López et al [42] presented a framework on the basis of opinion mining and semantic profiling on educational resource platform.…”

Weighted word embeddings and clustering‐based identification of question topics in MOOC discussion forum posts

“…In this contribution, the predictive performance of conventional classifiers (such as Naïve Bayes, support vector machines, and random forest) and conventional deep learning models (such as convolutional neural network, recurrent neural network, long short-term memory, and gated recurrent unit) has been evaluated on MOOC discussion forum posts. Lin et al [27] examined the predictive performance of lexiconbased and machine learning-based schemes for opinion mining on student evaluations of teaching. Recently, Onan [33] examined the predictive performance of machine learning models and deep learning architectures for opinion mining on students' evaluation of teaching.…”