Assisting Undergraduate Students in Writing Spanish Methodology Sections

Abstract

In undergraduate theses, a good methodology section should describe the series of steps that were followed in performing the research. To assist students in this task, we develop machine-learning models and an app that uses them to provide feedback while students write. We construct an annotated corpus that identifies sentences representing methodological steps and labels when a methodology contains a logical sequence of such steps. We train machine-learning models based on language modeling and lexical features that can identify sentences representing methodological steps with 0.939 f-measure, and identify methodology sections containing a logical sequence of steps with an accuracy of 87%. We incorporate these models into a Microsoft Office Addin, and show that students who improved their methodologies according to the model feedback received better grades on their methodologies.

A Survey of Current Machine Learning Approaches to Student Free-Text Evaluation for Intelligent Tutoring

“…2 An extension of Writing Mentor to Spanish has recently been released (Cahill et al, 2021). Also working on Spanish, the system by González-López et al (2020) specifically evaluates the methodology section of Mexican college students' theses in engineering subjects. Argumentation skills in the writings of German-language business administration students are the target of the AL system (Wambsganss et al, 2020), which evaluates, among others, the coherence and persuasiveness of students' argumentation and presents its findings in a dashboard view.…”

“…A regression model is typically used when a score is the desired output verdict (Dong et al, 2017;Mathias & Bhattacharyya, 2020). Conversely, classification is used when the model output is a correctness judgement (Leeman-Munk et al, 2014), or when the model is designed to recognise specific writing components in the student text (González-López et al, 2020).…”

“…As features are highly dependent on the concrete task and use case, the general features can be complemented by tailored feature sets that reflect systems' evaluation goals in specific use cases. To illustrate, González-López et al (2020) give feedback on methodology sections in college-level engineering theses, hence their feature set includes keywords that indicate the presence of a logical sequence of steps; Cahill et al (2020) score student responses to mathematics questions where the responses contain mathematical expressions, and therefore they include the correctness of those mathematical expressions as a feature for the feature-based scorer. Moreover, Nguyen and Litman (2018) and Ghosh et al (2016) find argumentation features to be useful for scoring persuasive student essays.…”

“…A major advantage of hand-crafted features is their human-interpretable nature such that analysing the features can yield interpretable insights: For instance, in their feature-based short-answer scoring task, Kumar et al (2020) extract the importance of each individual linguistically-informed feature set and use it as a basis for feedback to the student. Moreover, feature-based approaches are useful when little annotated training data is available (González-López et al, 2020); Nadeem et al (2019) additionally find that in a low-resource scenario, hand-crafted features can be particularly effective in combination with a neural architecture (see "Combination of Neural and Feature-based Models"). Finally, 's experiments on short-answer scoring in an adversarial setting suggest that classical, feature-based systems might be less susceptible to certain types of gaming and cheating attempts by students than end-to-end neural models are (see "Automatic Short-Answer Scoring (ASAS)").…”

Analysis and comparison of deep learning models for user stories identification

“…BERT uses a masked language modeling objective to pre-train the transformer network on an extensive unlabeled data [24]. In [25], it can be found an implementation and examples of the use of a module that fits this architecture trained in Wikipedia and BookCorpus.…”