An approach to designing computer-based evaluation of student constructed responses: Effects on achievement and instructional time

Gibbs, William J.; Peck, Kyle L.

doi:10.1007/bf02941040

Cited by 7 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, guessing or random marking may lead to unjust high scores (Funk, & Dickson, 2011). Generally, open answers are found to be more informative, because compared to conventional MC tests guessing is minimized and the correct solution cannot be derived by successive elimination (Gibbs, 1995). In a study about vocabulary learning, Heim and Watts found significant differences between multiplechoice tests and open-ended scores, the former being substantially higher than the latter.…”

Section: Multiple Choice Tests Versus Open Ended Questionsmentioning

confidence: 99%

Automated essay scoring in applied games: Reducing the teacher bandwidth problem in online training

Westera

Dascălu

Kurvers

et al. 2018

Computers & Education

View full text Add to dashboard Cite

This paper presents a methodology for applying automated essay scoring in educational settings. The methodology was tested and validated on a dataset of 173 reports (in Dutch language) that students have created in an applied game on environmental policy. Natural Language Processing technologies from the ReaderBench framework were used to generate an extensive set of textual complexity indices for each of the reports. Afterwards, different machine learning algorithms were used to predict the scores. By combining binary classification (pass or fail) and a probabilistic model for precision, a trade-off can be made between validity of automated score prediction (precision) and the reduction of teacher workload required for manual assessment. It was found from the sample that substantial workload reduction can be achieved, while preserving high precision: allowing for a precision of 95% or higher would already reduce the teacher's workload to 74%; lowering precision to 80% produces a workload reduction of 50%.

show abstract

Section: Multiple Choice Tests Versus Open Ended Questionsmentioning

confidence: 99%

Automated essay scoring in applied games: Reducing the teacher bandwidth problem in online training

Westera

Dascălu

Kurvers

et al. 2018

Computers & Education

View full text Add to dashboard Cite

show abstract

“…The educational value of such assessment instruments remains controversial. "The primary criterion for selection of a question type… should not be the ease with which the response to the question gets evaluated by the computer but rather the type of learning the question is designed to assess" (Gibbs and Peck, 1995). Certainly for information technology assessment, multiple choice and structured questions do not provide authentic performance assessment which requires "an active demonstration of the knowledge in question, as opposed to talking or writing about it" (Biggs, 1999).…”

Section: Automated Assessmentmentioning

confidence: 99%

Formative Automated Computer Testing (FACT)

Hunt

Hughes

Rowe

2002

Brit J Educational Tech

View full text Add to dashboard Cite

This study describes the context for the development of a tool to formatively assess information technology skills of students. The tool provides a reliable and valid assessment of word processing competency, utilizing automation to apply the test instrument via the Microsoft Office package. Tests can be designed directly by tutors, and delivered via a network. Evaluation of the tests suggests that immediate automated testing is preferred by students compared to a traditional written test. There is evidence that the tool improves the IT skills of its users, whereas a traditional written test has no such beneficial effect. Introduction and backgroundIn the United Kingdom, the Report of the National Committee of Inquiry into Higher Education in 1997 recognised the need for Higher Education institutions to provide a programme for all students which included information technology (IT) as one of a set of key skills. The Quality Assurance Agency was established as one mechanism for guiding institutions towards acceptable specifications of knowledge, understanding and skills for different programmes. The importance of information technology is demonstrated by its inclusion within the "transferable skills" component of subject benchmark statements published by the QAA. The set of information technology skills required tends to evolve in response to computing hardware and software developments, the changing set of skills brought by new entrants from school experience, and the expectations of tutors and professional bodies (Martin, 1997). Additionally, distinct disciplines have subject-specific requirements of IT. Nevertheless, the core components of most programmes include skills in word processing, email, information retrieval from databases and the Internet, and competence in the basic use of a personal computer. These skills are taught either by means of discrete IT classes or by embedding the teaching of IT skills at relevant points within subject-specific classes. The efficacy of the different approaches has been a popular topic for researchers (for example, see Martin, 1997). Less well researched is the most appropriate and effective method of assessing such skills.

show abstract

“…There has been a growing interest to use computer‐based applications in many areas of human's daily life during the last decade including education systems . This causes changes in workflows of many tasks that are performed manually in the past.…”

Section: Introductionmentioning

confidence: 99%

Process mining based approach to performance evaluation in computer‐aided examinations

Baykasoğlu

Özbel

Dudaklı

et al. 2018

Comp Applic In Engineering

View full text Add to dashboard Cite

A considerable part of educational systems tends to be online and computer oriented. However, online examination may create some difficulties during the evaluation of student performance. Process mining which arises as a new concept presents various powerful techniques for processing and analyzing different types of data by making use of some advanced information technologies. This paper proposes a novel approach based on process mining for evaluating the performance of students that should follow certain tasks on the computer. The proposed approach is composed of two main phases which are process mining and similarity analysis. Automatic assessment is performed totally in six steps in order to obtain students’ final grades. In addition, cheat control is possible in the last step thanks to the similarity analysis. A real‐life application in an Enterprise Resource Planning (ERP) course is performed in order to present usefulness, validity and practicality of the proposed approach. Furthermore, to evaluate the performance of the assessment system, we compared the assessment mechanism against instructor. A total of 15 students’ answers belonging to computer‐aided exam are assessed by instructor and the results showed a very good agreement between the automatic assessment system and instructor.

show abstract

An approach to designing computer-based evaluation of student constructed responses: Effects on achievement and instructional time

Cited by 7 publications

References 14 publications

Automated essay scoring in applied games: Reducing the teacher bandwidth problem in online training

Automated essay scoring in applied games: Reducing the teacher bandwidth problem in online training

Formative Automated Computer Testing (FACT)

Process mining based approach to performance evaluation in computer‐aided examinations

Contact Info

Product

Resources

About