Evaluating Machine Learning Approaches to Classify Pharmacy Students’ Reflective Statements

Abstract

Reflective writing is widely acknowledged to be one of the most effective learning activities for promoting students' self-reflection and critical thinking. However, manually assessing and giving feedback on reflective writing is time consuming, and known to be challenging for educators. There is little work investigating the potential of automated analysis of reflective writing, and even less on machine learning approaches which offer potential advantages over rule-based approaches. This study reports progress in developing a machine learning approach for the binary classification of pharmacy students' reflective statements about their work placements. Four common statistical classifiers were trained on a corpus of 301 statements, using emotional, cognitive and linguistic features from the Linguistic Inquiry and Word Count (LIWC) analysis, in combination with affective and rhetorical features from the Academic Writing Analytics (AWA) platform. The results showed that the Random-forest algorithm performed well (F-score=0.799) and that AWA features, such as emotional and reflective rhetorical moves, improved performance.

Evaluating Reflective Writing in Pre-Service Teachers: The Potential of a Mixed-Methods Approach

“…This approach is useful for exploratory research but needs a theoretical foundation for assessing the quality of reflection. In contrast, Cui et al [19] and Liu et al [42] form a new reflective element by consolidating selected linguistic indicators into a unified taxonomy of reflection quality based on theoretical frameworks such as Gibbs [29] and Boud et al [21]. However, a challenge in this approach is matching the linguistic indices with the reflective elements reasonably and progressively.…”

Framing Professional Learning Analytics as Reframing Oneself

“…The possibility for learning technologies to recognise and give feedback on such writing places this example firmly in the KAW right-side quadrants, but of the three examples we discuss, this is the least tested in professional learning contexts. Future work should investigate performance on different kinds of professional reflective writing, and explore the potential of machine learning, which has demonstrated potential in reflective writing [64], [69]. Such tools should be carefully piloted in workplaces, ideally as an integrated part of leadership development programs, and may find adoption most quickly in professions already familiar with this form of reflective practice, such as teaching, nursing, medicine or pharmacy.…”

Performance gains from adaptive eXtended Reality training fueled by artificial intelligence

“…96 Furthermore, providing mismatched metacognitive prompts to trainees with low prior knowledge that are struggling with high levels of extraneous processing within XR training systems would be expected to lead to confusion and a lack of learning outcome gains. 97 Adaptive metacognitive prompts can be implemented in XR training via a combination of natural language processing to analyze trainee's verbal protocols, statistical classifiers (e.g., random forests, support vector machine, naı ¨ve Bayes) to automatically code (classify) the protocols according to relevant schemes (e.g., cognitive, psychomotor, affective; noun, verb, adjective 98 ), and rule-based systems. 97,99 In addition, AI algorithms could be used to generate adaptive metacognitive prompts, such as a hybrid, fuzzy-neural processing system that incorporates genetic learning algorithms.…”