The Impact of Adding Textual Explanations to Next-step Hints in a Novice Programming Environment

Marwan, Samiha; Lytle, Nicholas; Williams, Joseph Jay; Price, Thomas W.

doi:10.1145/3304221.3319759

Cited by 30 publications

(15 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As such, these high-level interventions should be designed to avoid revealing too much information (e.g., correct solutions) so that the students can still get sufficient practice on the educational goals of the assignment. In cases where the students still can not make progress after seeing these initial interventions, a more bottom-out intervention, such as next-step action hint with text explanations [22] and worked examples with closer resemblance to a correct solution [37], may be needed to help a struggling student make progress to avoid unproductive struggling and frustration, which may lead to failing or dropping out of the class.…”

Section: Discussion 61 Implications Of Intervention Reasonsmentioning

confidence: 99%

“…Many programming learning environments provide on-demand support that struggling students can request to help them get unstuck and proceed [6,22,28,38]. However, research studies had shown that many novices exhibited poor help-seeking behaviorssome abused automated support to reach a correct solution faster, while some others avoided using automated support even when they needed them [19,30].…”

Section: Related Work 21 Identifying Struggling Studentmentioning

confidence: 99%

“…Next-step hints is a type of intervention that tells struggling students what the next actions are (e.g., insertion, deletion, replacement) to help them move closer to a correct solution [28]. Research showed that next-step hints with sufficient textual explanations improved student performance in programming assignments [21,22]. Breaking assignment requirements into subgoals has been used to provide positive feedback [20,23,33], which helped improve student performance and engagement in programming tasks.…”

Section: Types Of Interventionsmentioning

confidence: 99%

“…While these interventions can help students learn and advance during the assignments, they each have their challenges and drawbacks. For example, some students found next-step hints to be too specific and difficult to interpret without further explanation [22]. Also, it is challenging to create good worked examples that help students move forward without revealing too much information of the correct solution [37].…”

Section: Types Of Interventionsmentioning

confidence: 99%

See 3 more Smart Citations

You Really Need Help: Exploring Expert Reasons for Intervention During Block-based Programming Assignments

Dong

Shabrina

Marwan

et al. 2021

Proceedings of the 17th ACM Conference on International Computing Education Research

Self Cite

View full text Add to dashboard Cite

In recent years, research has increasingly focused on developing intelligent tutoring systems that provide data-driven support for students in need of assistance during programming assignments. One goal of such intelligent tutors is to provide students with quality interventions comparable to those human tutors would give. While most studies focused on generating different forms of on-demand support, such as next-step hints and worked examples, at any given moment during the programming assignment, there is a lack of research on why human tutors would provide different forms of proactive interventions to students in different situations. This information is critical to know to allow the intelligent programming environments to select the appropriate type of student support at the right moment.In this work, we studied human tutors' reasons for providing interventions during two introductory programming assignments in a block-based environment. Three human tutors evaluated a sample of 86 struggling moments identified from students' log data using a data-driven model. The human tutors specified whether and why an intervention was needed (or not) for each struggling moment. We analyzed the expert tags and their consensus discussions and extracted three main reasons that made the experts decide to intervene: "missing key components to make progress", "using wrong or unnecessary blocks", "misusing needed blocks", "having critical logic errors", "needing confirmation and next steps", and "unclear student intention". We use six case studies to illustrate specific student code trace examples and the tutors' reasons for intervention. We also discuss the potential types of automatic interventions that could address these cases. Our work sheds light on when and why students might need programming interventions. These insights contribute towards improving the quality of automated, data-driven support in programming learning environments.

show abstract

Section: Discussion 61 Implications Of Intervention Reasonsmentioning

confidence: 99%

Section: Related Work 21 Identifying Struggling Studentmentioning

confidence: 99%

Section: Types Of Interventionsmentioning

confidence: 99%

Section: Types Of Interventionsmentioning

confidence: 99%

See 2 more Smart Citations

You Really Need Help: Exploring Expert Reasons for Intervention During Block-based Programming Assignments

Dong

Shabrina

Marwan

et al. 2021

Proceedings of the 17th ACM Conference on International Computing Education Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several studies have evaluated specific system features that support program construction. These features include, for example, next-step hints and textual explanations [26,27], hints generated from the inputs entered to programs [12], and refactoring support in a visual programming environment [33]. Learner-generated written annotations have been used for constructing tutorials shown to other learners [14].…”

Section: System Evaluationsmentioning

confidence: 99%

Crowdsourcing in Computing Education Research: Case Amazon MTurk

Hellas

Zavgorodniaia

Sorva

2020

Koli Calling '20: Proceedings of the 20th Koli Calling International Conference on Computing Education Research

View full text Add to dashboard Cite

Crowdsourcing platforms such as Amazon MTurk provide access to a human workforce that can be given tasks to complete online for a fee. In this article, we review studies in computing education research (CER) that rely on crowdsourcing; we also describe our own experiences of using Amazon MTurk for a CER study. We discuss challenges in recruiting workers with specific backgrounds-such as no programming experience-and considerations in filtering out unreliable research participants. Combining recommendations from the literature with the lessons that we learned whilst conducting our study, we synthesize advice for researchers in CER who are considering crowdsourcing. In our case study, we did not find widespread foul play by crowdsourced workers and, overall, our experiences and the literature suggest that crowdsourced CER is feasible. It is, however, uncertain to what extent crowdsourced data can produce answers that apply to specific educational contexts. More research is needed before definitive conclusions can be drawn about the validity and generalizability of crowdsourced CER. CCS CONCEPTS• Social and professional topics → Computing education.

show abstract

Alloy Repair Hint Generation Based on Historical Data

Barros,

Neto,

Cunha

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Platforms to support novices learning to program are often accompanied by automated next-step hints that guide them towards correct solutions. Many of those approaches are data-driven, building on historical data to generate higher quality hints. Formal specifications are increasingly relevant in software engineering activities, but very little support exists to help novices while learning. Alloy is a formal specification language often used in courses on formal software development methods, and a platform—Alloy4Fun—has been proposed to support autonomous learning. While non-data-driven specification repair techniques have been proposed for Alloy that could be leveraged to generate next-step hints, no data-driven hint generation approach has been proposed so far. This paper presents the first data-driven hint generation technique for Alloy and its implementation as an extension to Alloy4Fun, being based on the data collected by that platform. This historical data is processed into graphs that capture past students’ progress while solving specification challenges. Hint generation can be customized with policies that take into consideration diverse factors, such as the popularity of paths in those graphs successfully traversed by previous students. Our evaluation shows that the performance of this new technique is competitive with non-data-driven repair techniques. To assess the quality of the hints, and help select the most appropriate hint generation policy, we conducted a survey with experienced Alloy instructors.

show abstract

The Impact of Adding Textual Explanations to Next-step Hints in a Novice Programming Environment

Cited by 30 publications

References 25 publications

You Really Need Help: Exploring Expert Reasons for Intervention During Block-based Programming Assignments

You Really Need Help: Exploring Expert Reasons for Intervention During Block-based Programming Assignments

Crowdsourcing in Computing Education Research: Case Amazon MTurk

Alloy Repair Hint Generation Based on Historical Data

Contact Info

Product

Resources

About