The automatic assessment of free text answers using a modified BLEU algorithm

Noorbehbahani, Fakhroddin; Kardan, Ahmad A.

doi:10.1016/j.compedu.2010.07.013

Cited by 64 publications

(29 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Experimental results showed that the M-BLEU method achieves the highest correlation with human expert scores compared to other assessment methods such as latent semantic analysis (LSA) (Grasser, Chipman, Haynes, and Olney, 2005) and n-gram co-occurrence (Noorbehbahani, and Kardan, 2011). However, the human score is significantly different from that of M-BLEU method if the student answer is too abstract, or a student answer is much longer than the reference answer because of unnecessary explanations.…”

Section: Introductionmentioning

confidence: 97%

“…Meanwhile, word co-occurrence may be rare or even null in short free-text CMC, and yet hassimilar meanings. Also, the use of a large corpus is impractical because of the unrestricted possible different free-text CMC's to a prescribed topic in online discussion (Noorbehbahani, and Kardan, 2011). An automatic assessment in recent study (Rodriguesand and Oliveira, 2014) was performed inboth types of exams, training and evaluation for short free-text CMC.…”

Section: Introductionmentioning

confidence: 99%

“…The M-BLEU algorithm presented in (Noorbehbahani, &Kardan, 2011) identifies the most similar reference answer to a student answer; a similarity score is calculated and applied to score the answers provided by students. Experimental results showed that the M-BLEU method achieves the highest correlation with human expert scores compared to other assessment methods such as latent semantic analysis (LSA) (Grasser, Chipman, Haynes, and Olney, 2005) and n-gram co-occurrence (Noorbehbahani, and Kardan, 2011).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Intelligent Short-text Auto-marking Algorithm for Text-based Computer-mediated Communication

Huang¹,

Chang²,

Guan³

et al. 2015

Journal of Information Technology and Application in Education

View full text Add to dashboard Cite

In this work, an intelligent short text auto-marking algorithm for text-based computer-mediated communication is proposed. After each learner issues a short text post on the online discussion platform, a novel feature extraction approach is adopted to derive the input parameters of a one-class Support Vector Machines (SVMs) classifier. The classifier then determines if the learners' posts are related to the concept maps previously outlined by the instructor. Meanwhile, a concept map analysis approachis used to assist in determining the correctness or completeness of the learners' concept. A feedback rule construction mechanism is used to issue feedback messages to learners if necessary. The experimental results revealed that the proposed approach achieved very good diagnosis results and verified its effectiveness.

show abstract

Section: Introductionmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Intelligent Short-text Auto-marking Algorithm for Text-based Computer-mediated Communication

Huang¹,

Chang²,

Guan³

et al. 2015

Journal of Information Technology and Application in Education

View full text Add to dashboard Cite

show abstract

“…Rodrigues and Oliveira [17] compute a cosine similarity after a preprocess. A modification of the BLEU algorithm [18] is used by Noorbehbahani and Kardan [19]. An ensemble of Latent Semantic Analysis (LSA) and n-gram co-ocurrences is the method proposed by He et al [20].…”

Section: Related Workmentioning

confidence: 99%

A factorization approach to evaluate open-response assignments in MOOCs using preference learning on peer assessments

Luaces¹,

Díez²,

Alonso‐Betanzos³

et al. 2015

Knowledge-Based Systems

View full text Add to dashboard Cite

Evaluating open-response assignments in Massive Open Online Courses is a difficult task because of the huge number of students involved. Peer grading is an effective method to address this problem. There are two basic approaches in the literature: cardinal and ordinal. The first case uses grades assigned by student-graders to a set of assignments of other colleagues. In the ordinal approach, the raw materials used by grading systems are the relative orders that graders appreciate in the assignments that they evaluate. In this paper we present a factorization method that seeks a trade-off between cardinal and ordinal approaches. The algorithm learns from preference judgments to avoid the subjectivity of the numeric grades. But in addition to preferences expressed by student-graders, we include other preferences: those induced from assignments with significantly different average grades. The paper includes a report of the results obtained using this approach in a real world dataset collected in 3 Universities of Spain, A Coruña, Pablo de Olavide at Sevilla, and Oviedo at Gijón. Additionally, we studied the sensitivity of the method with respect to the number of assignments graded by each student. Our method achieves similar or better scores than staff instructors when we measure the discrepancies with other instructor's grades. * Corresponding author: Tel: +34 985 182 028

show abstract

“…Az elektronikus tesztelés fejlődésének egyik fontos pontja, hogy az automatizált értékelési technikák a hosszabb szöveges válaszok esetében sokat fejlődnek (Noorbehbahani és Kardan, 2011;He, Hui és Quan, 2009idézi Redecker és Johannessen, 2013. Az automatizált értékelés drámaian csökkenti az idő és költség igényét az olyan komplex képességek mérésének esetén, mint a szövegalkotás.…”

Section: áBra Az Elektronikus Tesztelés Generációiunclassified

Áttérés a technológia alapú mérés-értékelésre:

Hülber

View full text Add to dashboard Cite

The automatic assessment of free text answers using a modified BLEU algorithm

Cited by 64 publications

References 3 publications

An Intelligent Short-text Auto-marking Algorithm for Text-based Computer-mediated Communication

An Intelligent Short-text Auto-marking Algorithm for Text-based Computer-mediated Communication

A factorization approach to evaluate open-response assignments in MOOCs using preference learning on peer assessments

Áttérés a technológia alapú mérés-értékelésre:

Contact Info

Product

Resources

About