Evaluating a new exam question

Denny, Paul; Luxton-Reilly, Andrew; Simon, Beth

doi:10.1145/1404520.1404532

Cited by 177 publications

(88 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to online learning environments, Parson's puzzles can be used in traditional paper exams where coding exercises (i.e., programming with a pen and a paper) are often problematic. Interestingly, in the context of paper exams, points from Parson's puzzles correlate well with open ended code writing question (Denny, Luxton-Reilly, & Simon, 2008). Moreover, in the same study, points from neither of these question types correlated with points from tracing exercises.…”

Section: Two-dimensional Parson's Puzzlesmentioning

confidence: 67%

“…In this approach, distractors can be designed to separate problems in problem solving and in syntax. According to Denny et al (2008), grouping reduces cognitive load not relevant to problem solving or to programming.…”

Section: Different Types Of Puzzlesmentioning

confidence: 99%

“…This gives a lot more freedom and, if combined with distractors without line grouping, exercises can get too difficult (Denny et al, 2008).…”

Section: Different Types Of Puzzlesmentioning

confidence: 99%

See 2 more Smart Citations

Two-Dimensional Parson’s Puzzles: The Concept, Tools, and First Observations

Ihantola¹,

Karavirta

2011

JITE:IIP

View full text Add to dashboard Cite

Executive SummaryParson's programming puzzles are a family of code construction assignments where lines of code are given, and the task is to form the solution by sorting and possibly selecting the correct code lines. We introduce a novel family of Parson's puzzles where the lines of code need to be sorted in two dimensions. The vertical dimension is used to order the lines, whereas the horizontal dimension is used to change control flow and code blocks based on indentation as in Python. Python blocks have no explicit begin/end statements or curly braces to mark where the block starts or stops. Instead, indentation is used to define starts and stops of blocks and functions.In addition, we introduce tools supporting two-dimensional Parson's puzzles: (1) MIT licensed JavaScript widget to embed our puzzles to any HTML, and (2) server to create, share, and solve puzzles.We have observed how experienced programmers solve our puzzles. Such users often start by dragging the method signature to the beginning and continue by defining majority of the control flow (i.e., loop statements, assignments, conditional statements). Only after these are done, details, including initialization of variables and handling of corner cases, are dragged to correct positions in the middle of the previously structured code. This shows that even experts are not able to solve puzzles linearly, i.e., line by line, starting from the first. Thus, user interfaces (UIs) should minimize the work needed to insert a line between two adjacent lines of existing code. In some of the existing Parson's Puzzle UIs this is not the case.Another observation we made is that too often users don't ask or use automated feedback. Why this happens needs further investigations. Perhaps experienced users are too proud to ask a tool to help them (especially when being observed), or perhaps users don't recognize when they are stuck and should ask for help. Providing constant feedback is one way to tackle this problem. However, the obvious downside of the constant feedback is that solving an exercise can become trial-and-error repetition.

show abstract

Section: Two-dimensional Parson's Puzzlesmentioning

confidence: 67%

Section: Different Types Of Puzzlesmentioning

confidence: 99%

See 1 more Smart Citation

Two-Dimensional Parson’s Puzzles: The Concept, Tools, and First Observations

Ihantola¹,

Karavirta

2011

JITE:IIP

View full text Add to dashboard Cite

show abstract

“…These assessments were designed to help learners develop familiarity with code tracing -the ability to read/understand code (Bornat, 1987;Lister, Fidge, & Teague, 2009;Lopez, Whalley, Robbins, & Lister, 2008). Inspired by Parson's puzzles (Denny, Luxton-Reilly, & Simon, 2008;Parsons & Haden, 2006), some quiz questions also involved presenting jumbled blocks in Scratch required for a program and having students snap them in correct order.…”

Section: Fact's Systems Of Assessmentsmentioning

confidence: 99%

Designing for deeper learning in a blended computer science course for middle school students

Grover

Pea

Cooper

2015

Computer Science Education

329

148

View full text Add to dashboard Cite

The focus of this research was to create and test an introductory computer science course for middle school. Titled "Foundations for Advancing Computational Thinking" (FACT), the course aims to prepare and motivate middle school learners for future engagement with algorithmic problem solving. FACT was also piloted as a seven-week course on Stanford's OpenEdX MOOC platform for blended in-class learning. Unique aspects of FACT include balanced pedagogical designs that address the cognitive, interpersonal, and intrapersonal aspects of "deeper learning"; a focus on pedagogical strategies for mediating and assessing for transfer from block-based to text-based programming; curricular materials for remedying misperceptions of computing; and "systems of assessments" (including formative and summative quizzes and tests, directed as well as open-ended programming assignments, and a transfer test) to get a comprehensive picture of students' deeper computational learning. Empirical investigations, accomplished over two iterations of a design-based research effort with students (aged 11-14 years) in a public school, sought to examine student understanding of algorithmic constructs, and how well students transferred this learning from Scratch to text-based languages. Changes in student perceptions of computing as a discipline were measured. Results and mixed-method analyses revealed that students in both studies (1) achieved substantial learning gains in algorithmic thinking skills, (2) were able to transfer their learning from Scratch to a text-based programming context, and (3) achieved significant growth toward a more mature understanding of computing as a discipline. Factor analyses of prior computing experience, multivariate regression analyses, and qualitative analyses of student projects and artifact-based interviews were conducted to better understand the factors affecting learning outcomes. Prior computing experiences (as measured by a pretest) and math ability were found to be strong predictors of learning outcomes.

show abstract

“…Students must select the correct lines of code and place them in the most appropriate order. It is hypothesized that Parson's problems provide useful insights into the transition between tracing and code writing skills as placing code in the correct order shows one type of understanding whilst identifying the correct syntax another type of comprehension (Denny, Luxton-Reilly, & Simon, 2008).…”

Section: The Local Impact Of Braceletmentioning

confidence: 99%

Wearing the Assessment ‘BRACElet’

Tan

Venables

2010

JITE:IIP

View full text Add to dashboard Cite

Executive SummaryThere exists a wealth of computing education literature devoted to interventions designed to overcome novices' difficulties in learning to write computer programs. However, various studies have shown that the majority of students at the end of a semester of instruction are still unable to write a simple computer program, despite the best efforts of their teachers (Lister et al., 2004;McCracken et al., 2001;Soloway, Bonar, & Ehrlich, 1983). In an effort to address this problem, a workshop titled Building Research in Australasian Computing Education (BRACE) was convened in 2004. BRACE brought together academics interested in learning and applying the techniques and methodologies of action research to the problem of poor student code-writing performance. At this workshop, and at those that followed, participants agreed to use end-ofsemester assessments to try to pinpoint the key steps and difficulties beginners faced in learning introductory programming. Subsequently the group, which has come to be known as the BRACElet project, has continued to meet twice a year in an effort to better understanding of how students learn and grasp various programming concepts.Over the past five years, the BRACElet project has fostered a multi-institutional community of practice amongst academics who teach novice computer programming. For participants, the BRACElet project provides the benefits of external moderation, quality assurance, and benchmarking of examinations. At a BRACElet meeting, attendees decide upon a small common set of question types for use within their own local examinations; it is the subsequent analysis and discussion of locally collected data from various institutions that comprises the BRACElet project's iterative series of action research. By examining across institutions common difficulties students may have on various question types, such as reading, tracing, and coding, the BRACElet project attempts to build theory on how learners acquire programming knowledge.In 2008, academics from Victoria University joined the BRACElet group. After modifying our end of semester examination to contain the BRACElet core question types, the examination was run and data was collected across the cohort of 32 students. The analysis treated student performance on code writing questions as the dependent variable and looked at the relationship between code writing questions and non-code writing questions. Overall, our study found that the combination of tracing and explaining questions, more so than each skill independently correlates highly to code writing skills supporting the

show abstract

Evaluating a new exam question

Cited by 177 publications

References 7 publications

Two-Dimensional Parson’s Puzzles: The Concept, Tools, and First Observations

Two-Dimensional Parson’s Puzzles: The Concept, Tools, and First Observations

Designing for deeper learning in a blended computer science course for middle school students

Wearing the Assessment ‘BRACElet’

Contact Info

Product

Resources

About