Improving Academic Decision-Making through Course Evaluation Technology

Marks, Adam; AL-Ali, Maytha; Majdalawieh, Munir; Bani-Hani, Anoud

doi:10.3991/ijet.v12i11.6987

Cited by 8 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers refer to course evaluations in a variety of ways, including student evaluation of teaching (SET) (Fogarty et al, 2013;Reisenwitz, 2015); end-of-course evaluations (EOCE) (Jaquett, et al,2017;Marks et al, 2017); end-of-course critiques (EOCC) (Morrison, 2011) or course evaluations (Kuch & Roberts, 2018;Perrett, 2013). Throughout this paper, we use the term course evaluation to describe the process of garnering student-provided feedback (paper-pencil or online) at the end of a term.…”

Section: Course Evaluationsmentioning

confidence: 99%

Examining Online Course Evaluations and the Quality of Student Feedback

Plante¹,

LeSage²,

Kay³

2022

JEdInfo

View full text Add to dashboard Cite

The purpose of this article was to provide a comprehensive review of research on the quality of student feedback from post-secondary institutions using online course evaluations versus traditional paper-pencil methods. Nineteen peer-reviewed articles published from 2000 to 2020 were examined for changes to course evaluations following a transition to online collection methods. Three themes emerged from the literature: effects on response rates, presence of non-response bias, and effects on comment quality. Results suggest that using online methods for collecting student feedback tends to decrease response rates somewhat, however, the effect is often temporary. Further, using online methods generated conflicting results on the presence of a non-response bias in open-ended comments with online methods. Many studies demonstrated that online methods increase the word counts in student-provided comments and that the constructive nature of the comments improved as well. The results may inform teaching and policy decisions as more institutions transition to online course evaluation collection methods, particularly given the restrictions imposed by the current COVID-19 crisis. Suggestions for future research include examining the usability of comments as well as trends in student feedback quality following the transition to emergency remote teaching during the global pandemic.

show abstract

Section: Course Evaluationsmentioning

confidence: 99%

Examining Online Course Evaluations and the Quality of Student Feedback

Plante¹,

LeSage²,

Kay³

2022

JEdInfo

View full text Add to dashboard Cite

show abstract

“…Marks (2017) said that education process which includes the effectiveness of learning will affects the quality of learning and increases learning achievement by factors such as course recommendations, student skills, and behavior detection [4].…”

Section: Surveys Carried Out Bymentioning

confidence: 99%

A Novel Approach to Rank Text-based Essays using Pagerank Method Towards Student’s Motivational Element

Arifin¹,

Che²,

Suryana³

2019

IJACSA

View full text Add to dashboard Cite

Learning outcomes is one of the important factors to measure student achievement during the learning process. Today's learning is more focused on problem-solving and reasoning to existing problems than an ordinary problem. Most exams have been directed to analysis questions for students to think and synthesize. As such is troublesome for most students, they are not ready to answer the question, thus, their answers almost similar to their friends. This implies that the teacher has tried to guide students to work professionally and originally. However, the Teacher facing difficulty assessing student's work, especially if the assignments are conducted online without faceto-face instructions/discussions. To bridge such a gap, the teacher needs a method or algorithm to measure their rank to encourage students making an original answer. This research provides a solution in calculating students' ranks based on the similarity score of the essay answers. Pagerank is a ranking algorithm used by Google, this algorithm utilizes a Markov matrix that contains the direction of similarity score for each student. These scores are computed by the power method until converging. Rank is displayed to the teacher to review the similarity level of students' answers. As such is presented a line chart in which the x-axis refers to the students and the y-axis depicts the level of similarity. Ranking computation in Matlab produces an Eigen vector which acts as the rank measure. The higher the rank, the more similar is their answers to others. Hence, students with high ranks to work their answers more seriously ensure their original thoughts. In conclusion, the similarity score matrix using the PageRank algorithm can contribute to the teacher in providing peer motivation and encouraging student's internal motivation by presenting the ranked-answers presentation.

show abstract

“…The history of education surveys has seen examination of the pros and cons, with some noting the subjectivity of students' perspectives and others pointing to the difficulties in gathering responses unbiased by low response rates and the challenges with analyzing qualitative survey responses in an objective way (McGourty et al, 2002;Shah & Pabel, 2019;Spooren et al, 2013;Stowell et al, 2012;Wallace et al, 2019;Wongsurawat, 2011). However, evidence has supported the validity of surveys, including qualitative and mixed methods approaches, for guiding changes to teaching practice and course design (Ferren & Aylesworth, 2001;Johnson & Onwuegbuzie, 2004;Lattuca & Domagal-Goldman, 2007;Marks et al, 2017;McKeachie, 1997;Mentkowski, 1991;Spooren et al, 2013).…”

Section: Introductionmentioning

confidence: 99%

A Large Language Model Approach to Educational Survey Feedback Analysis

Parker,

Anderson,

Stone

et al. 2024

Int J Artif Intell Educ

View full text Add to dashboard Cite

This paper assesses the potential for the large language models (LLMs) GPT-4 and GPT-3.5 to aid in deriving insight from education feedback surveys. Exploration of LLM use cases in education has focused on teaching and learning, with less exploration of capabilities in education feedback analysis. Survey analysis in education involves goals such as finding gaps in curricula or evaluating teachers, often requiring time-consuming manual processing of textual responses. LLMs have the potential to provide a flexible means of achieving these goals without specialized machine learning models or fine-tuning. We demonstrate a versatile approach to such goals by treating them as sequences of natural language processing (NLP) tasks including classification (multi-label, multi-class, and binary), extraction, thematic analysis, and sentiment analysis, each performed by LLM. We apply these workflows to a real-world dataset of 2500 end-of-course survey comments from biomedical science courses, and evaluate a zero-shot approach (i.e., requiring no examples or labeled training data) across all tasks, reflecting education settings, where labeled data is often scarce. By applying effective prompting practices, we achieve human-level performance on multiple tasks with GPT-4, enabling workflows necessary to achieve typical goals. We also show the potential of inspecting LLMs’ chain-of-thought (CoT) reasoning for providing insight that may foster confidence in practice. Moreover, this study features development of a versatile set of classification categories, suitable for various course types (online, hybrid, or in-person) and amenable to customization. Our results suggest that LLMs can be used to derive a range of insights from survey text.

show abstract

Improving Academic Decision-Making through Course Evaluation Technology

Cited by 8 publications

References 9 publications

Examining Online Course Evaluations and the Quality of Student Feedback

Examining Online Course Evaluations and the Quality of Student Feedback

A Novel Approach to Rank Text-based Essays using Pagerank Method Towards Student’s Motivational Element

A Large Language Model Approach to Educational Survey Feedback Analysis

Contact Info

Product

Resources

About