Automated Feedback Can Improve Hypothesis Quality

Abstract

Stating a hypothesis is one of the central processes in inquiry learning, and often forms the starting point of the inquiry process. We designed, implemented, and evaluated an automated parsing and feedback system that informed students about the quality of hypotheses they had created in an online tool, the hypothesis scratchpad. In two pilot studies in different domains ("supply and demand" from economics and "electrical circuits" from physics) we determined the parser's accuracy by comparing its judgments with those of human experts. A satisfactory to high accuracy was reached. In the main study (in the "electrical circuits" domain), students were assigned to one of two conditions: no feedback (control) and automated feedback. We found that the subset of students in the experimental condition who asked for automated feedback on their hypotheses were much more likely to create a syntactically correct hypothesis than students in either condition who did not ask for feedback.

Automated Assessment of and Feedback on Concept Maps During Inquiry Learning

“…The control condition included the concept mapping tool without feedback, whereas the experimental condition included a version of the concept mapping tool with feedback. In earlier work [35], we found that roughly half of students who had feedback available chose not to use it. We therefore assigned roughly two-thirds of students to the experimental condition, with the expectation that about half of all students in the experimental condition would actually use feedback.…”

“…The roughly equal distribution of students across these three groups shown in Table I shows that this approach was successful. In previous work [35], we found that letting students explore in a similar learning environment on their own took too much time, and some instruction on how to use the environment was needed. Students were therefore given an oral description of the apps, labs, and resources available to them in the learning environment at the beginning of each phase, and directed to wait for further instructions after completion of each phase.…”

“…Does automated feedback on concept maps improve the quality of hypotheses? Hypothesis quality was determined with the criteria described in [35], using a newly created grammar for each of the two domains involved [37]. As above, quality of hypotheses was compared between the control and experimental groups, and further contrasts were made between students who used feedback and those who did not, and between students who did not to use feedback when it was available to them, and those for whom feedback was not available.…”

“…The quality of hypotheses was scored using an automated tool developed in previous research [35]. Neither the effect of being in the experimental group ( = −0.09, = 0.3, = [0.51,1.64], = .767), nor that of using feedback ( = −0.14, = 0.4, = [0.4,1.91], = .734) was significant.…”

A Grammar for Hypothesis-Driven Visual Analysis

“…To that end, this paper proposes the first mechanism for describing and transforming visual analytics hypotheses into scientific hypotheses. Drawing from prior work in science and education literature [22,37,58], we propose a grammar for expressing hypothesisbased analytic questions. We start with a grammar that expresses a scientific hypothesis-a specific, narrow, and testable question-and relax it to express sets of scientific hypotheses.…”