Computer-assisted assessment of free-text answers

Marín, Diana Pérez; Pascual-Nieto, Ismael; Rodríguez, Pilar

doi:10.1017/s026988890999018x

Cited by 36 publications

(19 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Matching methods compare the students' answer against some reference (ideal answer) or template; [18] made a detailed survey of published algorithms using this paradigm.…”

Section: Related Workmentioning

confidence: 99%

Including Content-Based Methods in Peer-Assessment of Open-Response Questions

Luaces

Díez

Alonso-Sanz

et al. 2015

2015 IEEE International Conference on Data Mining Workshop (ICDMW)

View full text Add to dashboard Cite

Abstract-Massive Open Online Courses (MOOCs) are attracting the attention of a huge number of students all around the world. These courses include different types of assignments in order to evaluate the student's knowledge. However, these assignments are designed to allow a straightforward automatic evaluation. But, in this way it is not possible to evaluate some skills that would require answering open-response questions. Peerassessment (the students are asked to assess other assignments), is an effective method to overcome the impossibility of having staff graders for this task. Additionally, students gain a deeper knowledge about the subject of the assignments that they have to read critically. However, the grades given by student-graders must be filtered to avoid bias due to a lack of experience in assessment tasks. There are a number of approaches to do this. In this paper we present a factorization approach that in addition to the grades given by graders is able to incorporate a representation of the contents of the responses given by students using a Vector Space Model of the assignments. So we fill the gap between peer-assessment and content-based methods that use a shallow linguistic processing. The paper includes a report of the results obtained using this approach in a real world dataset collected in 3 universities of Spain, A Coruña, Pablo de Olavide at Sevilla, and Oviedo at Gijón. The scores obtained by the method presented here were compared with those provided by the staff of these universities. We report a considerable improvement whenever we use the content-based approach. In any case, we conclude that there is no evidence that staff grading would have led to more accurate grading outcomes than the assessment produced by our models.

show abstract

“…Matching methods compare the students' answer against some reference (ideal answer) or template; [18] made a detailed survey of published algorithms using this paradigm.…”

Section: Related Workmentioning

confidence: 99%

Including Content-Based Methods in Peer-Assessment of Open-Response Questions

Luaces

Díez

Alonso-Sanz

et al. 2015

2015 IEEE International Conference on Data Mining Workshop (ICDMW)

View full text Add to dashboard Cite

show abstract

“…As mentioned in Section 2, a few approaches have been proposed in the last years for supporting students' assessment, although important differences should be pointed out. A review of the state of the art in this field is presented in [49]. There, the conclusion is that there is still a long way to go before reaching the ideal system, although there are already some interesting systems.…”

Section: Discussionmentioning

confidence: 99%

Semantic Web Technologies for supporting learning assessment

Castellanos–Nieves

Fernández-Breis

Valencia‐García

et al. 2011

Information Sciences

View full text Add to dashboard Cite

“…So the performance of LSA and human graders is comparable. Perez et al [23], [24] developed a system using latent semantic analysis (LSA) and BLEU (bilingual evaluation understudy) algorithm to essay evaluation. LSA performs semantic analysis and modified BLEU as used by the authors performs syntactic analysis.…”

Section: Review Of Related Workmentioning

confidence: 99%

Multiagent System for Management and Evaluation of Examinations

Mittal¹

2017

ijarcs

View full text Add to dashboard Cite

Several efforts have been made to computerize Examination System. Agents are autonomous software components capable of making decision on behalf of other entities to realize desired goals. A Multiagent System is a collection of agents that cooperate to solve a problem. In this paper, a Multiagent System is designed, implemented and tested for Management and Evaluation of Examination (MASMEE). Examination can be subjective, objective or practical based on programming languages. The functions of paper setting, evaluation of answers' and result compilation are performed by intelligent agents. MASMEE mainly consists of two agents: Examiner Agent (EA) and Student Agent (SA). EA sets the paper, evaluates the submitted answers and compiles the results. The SA enables the students' to take the examination by providing question paper and collecting the answers.The paper setting is performed using a database of questions. A web based interface is developed for conduct of examination. The evaluation of subjective and objective examinations is performedbased on techniquesof information retrieval and Ontology.Practical evaluation is done using 4-stage framework, evaluating on the basis of compilation, functional testing, metrics, efficiencyand similarity of programs to standard program. The MASMEE system is implemented using Java andJava Agent Development Environment (JADE).Testing is performed by conducting tests with post-graduate students. The scores generated by MASMEE are compared with scores assigned by human examiner using correlation and are found to be satisfactory. It is concluded that MASMEE system can help to reduce the effort of human examiner.

show abstract

Computer-assisted assessment of free-text answers

Cited by 36 publications

References 29 publications

Including Content-Based Methods in Peer-Assessment of Open-Response Questions

Including Content-Based Methods in Peer-Assessment of Open-Response Questions

Semantic Web Technologies for supporting learning assessment

Multiagent System for Management and Evaluation of Examinations

Contact Info

Product

Resources

About