From Free Association to Goal-directed Problem-solving—Network Analysis of Students’ Use of Chemical Concepts in Mechanistic Reasoning

Asmussen, Gyde; Rodemer, Marc; Eckhard, Julia; Bernholt, Sascha

doi:10.1039/9781839167782-00090

Cited by 4 publications

(1 citation statement)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent trends in the chemistry education research literature indicate the value of supporting student reasoning through eliciting written responses to open-ended questions that require engagement with conceptual knowledge. , For example, studies call for the use of writing-to-learn (WTL) assignments, − constructed response items, − or case-comparison activities, − all of which can be used to elicit written responses from students. The pedagogical motivation behind these tasks is to require students to solve problems and engage in reasoning by considering underlying conceptual ideas rather than relying on memorization or other rote learning strategies.…”

Section: Introductionmentioning

confidence: 99%

Comparing Student and Generative Artificial Intelligence Chatbot Responses to Organic Chemistry Writing-to-Learn Assignments

Watts,

Dood,

Shultz

et al. 2023

J. Chem. Educ.

View full text Add to dashboard Cite

Chemistry education research demonstrates the value of open-ended writing tasks, such as writing-to-learn (WTL) assignments, for supporting students’ learning with topics including reasoning about reaction mechanisms. The emergence of generative artificial intelligence (AI) technology, such as chatbots ChatGPT and Bard, raises concerns regarding the value of open-ended writing tasks in the classroom; one concern involves academic integrity and whether students will use these chatbots to produce sufficient responses to open-ended writing tasks. The present study investigates the degree to which generative AI chatbots exhibit mechanistic reasoning in response to organic chemistry WTL assignments. We produced responses from three generative AI chatbots (ChatGPT-3.5, ChatGPT-4, and Bard) to two WTL assignments developed to elicit students’ mechanistic reasoning. Using previously reported machine learning models for analyzing student writing in response to the WTL assignments, we analyzed the chatbot responses for the inclusion of features pertinent to mechanistic reasoning. Herein, we report quantitative analyses of (1) the differences between chatbot responses on the two assignments and (2) the differences between chatbot and authentic student responses. Findings indicate that chatbots respond differently to different WTL assignments. Additionally, the chatbots rarely incorporated the discussion of electron movement, a key feature of mechanistic reasoning. Furthermore, the chatbots, in general, do not engage in mechanistic reasoning at the same level as students. We contextualize the results by considering academic integrity with the assumption that students’ intentions are to engage in academically honest behavior, and we focus on understanding the ethical uses of generative AI for classroom assignments.

show abstract

Section: Introductionmentioning

confidence: 99%

Comparing Student and Generative Artificial Intelligence Chatbot Responses to Organic Chemistry Writing-to-Learn Assignments

Watts,

Dood,

Shultz

et al. 2023

J. Chem. Educ.

View full text Add to dashboard Cite

show abstract

Patterns in Explanations of Organic Chemistry Reaction Mechanisms: A Text Analysis by Level of Explanation Sophistication

Crowder,

Raker

2024

J. Chem. Educ.

View full text Add to dashboard Cite

Learning the language of organic chemistry, i.e., how to describe reaction mechanisms, is crucial to success in any postsecondary organic chemistry course. However, it is well-known that learners struggle with reasoning about and explaining reaction mechanisms beyond surface-level features. Multiple studies have sought to aid learners in developing these skills. Investigating the connections that learners make regarding reaction mechanisms through their explanations provides insight into how we can better promote the development of learners' reasoning skills. In this study, we evaluate 20,000+ learner explanations of 90 reaction mechanisms. We use network analysis to explore patterns in keywords used by learners and visualize the word connections between them, based on their co-occurrence, within our entire data set, by reaction type, and by levels of explanation sophistication. Our results indicate that learners consistently rely on explicit surface-level features in their explanations with expected contextual variance by reaction type. This trend persists across the levels of sophistication, however, with improvements in the use of vocabulary and coherency as sophistication progresses. We hypothesize that this is evidence of learners actively working toward constructing understanding as they experiment with and refine their vocabulary until they are able to pare down their explanations in a coherent manner. This work offers insights for instructors seeking to promote the development of learners' reasoning skills and for researchers interested in the development of machine-learning models to assist in evaluating learner explanations of reaction mechanisms.

show abstract

Blooming student difficulties in dealing with organic reaction mechanisms – an attempt at systemization

Asmussen

Rodemer

Bernholt

2023

Chem. Educ. Res. Pract.

Self Cite

View full text Add to dashboard Cite

Students are known to have various difficulties in dealing with organic reaction mechanisms. A systematic classification of these difficulties appears necessary to design appropriate support. This paper presents insights into whether and how Bloom's revised taxonomy can be used to classify student difficulties in dealing with organic reaction mechanisms. We conducted an interview study with 12 undergraduate chemistry students using problem-solving tasks on nucleophilic substitution and elimination reactions to provide examples to test the classification. In our attempt at systemization, student difficulties are perceived as unachieved learning objectives. The classification reveals that student difficulties pertain to different cognitive process and knowledge dimensions. Specific major difficulties occurred within each cell of Bloom's revised taxonomy and for individual students. Our analysis suggests that general support for dealing with reaction mechanisms might be less beneficial for some students and that more adapted support is needed. Our approach of using Bloom's revised taxonomy to classify student difficulties might also benefit other domains to better understand student difficulties and evaluate appropriate support.

show abstract

From Free Association to Goal-directed Problem-solving—Network Analysis of Students’ Use of Chemical Concepts in Mechanistic Reasoning

Cited by 4 publications

References 49 publications

Comparing Student and Generative Artificial Intelligence Chatbot Responses to Organic Chemistry Writing-to-Learn Assignments

Comparing Student and Generative Artificial Intelligence Chatbot Responses to Organic Chemistry Writing-to-Learn Assignments

Patterns in Explanations of Organic Chemistry Reaction Mechanisms: A Text Analysis by Level of Explanation Sophistication

Blooming student difficulties in dealing with organic reaction mechanisms – an attempt at systemization

Contact Info

Product

Resources

About