Toward Automatic Inference of Causal Structure in Student Essays

Abstract

Abstract. With an increasing focus on science and technology in education comes an awareness that students must be able to understand and integrate scientific explanations from multiple sources. As part of a larger project aimed at deepening our understanding of student processes for integrating multiple sources of information, we are developing machine learning and natural language processing techniques for evaluating students' argumentative essays. In previous work, we have focused on identifying conceptual elements of the essays. In this paper, we present a method for inferring the causal structure of student essays. We used a standard parser to derive grammatical dependencies of the essay and converted them to logic statements. Then a simple inference mechanism was used to identify concepts linked to syntactic connectors by these dependencies. The results suggest that we will soon be able to provide explicit feedback that enables teachers and students to improve comprehension.

Different Approaches to Assessing the Quality of Explanations Following a Multiple-Document Inquiry Activity in Science

“…The second system was focussed upon the argument structure present within each student's essay and scored the essays for the presence of causal chains of ideas that culminated in recent changes in global temperatures. (For similar work using this approach see Hastings et al, 2014). Since both of these approaches are based on the a priori causal model created by the document set, only ideas and concepts present in the documents received credit for each approach.…”

“…In contrast with Hastings et al (2014) which used hand-coded essays to determine concepts directly, in this study we used the output of the concept detection from the automatic inference mechanism described above as input for automated detection of causal connections.…”

Towards mining sequences and dispersion of rhetorical moves in student written texts

“…Obviously the syntactic dependencies, represented as sequences of parts of speech in essays, can also be used to infer structures -for example, causal relations [18], or term-definition pairs [14] present in a text, with the additional analysis of temporal references in a text supporting identification of question-answer pairs [2], and types of discourse [38]. Association rule and sequence mining approaches have been similarly used to identify relationships between constructs in a text [for example, 1,20,35], and to detect erroneous sentences [42].…”

Machine Learning for Holistic Evaluation of Scientific Essays

“…We did not remove stop words as they can aid classification accuracy when tagging words based on their context. In previous work [11] we compared the performance of Linear Discriminant Analysis, an SVM, Random Forest and a Decision Tree on this task and found the SVM to be superior in terms of F 1 score, and so that technique was used in this paper.…”

“…In prior work, we have used tools to automatically identify core concepts [10,12,14], and, starting with human scoring of core concepts, automatically identify causal chains [11]. As the logical next step, we identified four levels of explanation quality that capture general goals for an explanation (e.g., accuracy, completeness, coherence).…”