Imperfect Answers in Multiple Choice Questionnaires

Díaz, Javier; Rifqi, Maria; Bouchon‐Meunier, Bernadette; Jhean-Larose, Sandra; Denhière, Guy

doi:10.1007/978-3-540-87605-2_17

Cited by 9 publications

(12 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The evaluation process of MCQs have been dealt by many researchers [10,11,[13][14][15]24,25,27]. Karen Scouller in ref.…”

Section: Related Workmentioning

confidence: 99%

“…Diaz et al in ref. have used the information gathered for Imperfect Answers through Ev‐MCQs to build better student models. The marked answers through Ev‐MCQs were able to capture the confidence of respondents but the whole procedure of marking answers is a tedious process.…”

Section: Related Workmentioning

confidence: 99%

“…Multiple‐choice questions (MCQs) are the most preferred format for evaluating the understanding and knowledge of a large number of candidates with respect to a specific context . An MCQ examination provides candidates with a set of objective questions that have more than one options as probable answer, out of which only one is correct.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

C‐BEAM: A confidence‐based evaluation of MCQs for providing feedback to instructors

Aleem

Gore

2018

Comp Applic In Engineering

View full text Add to dashboard Cite

Multiple‐choice questions (MCQs) examination is the most preferred methodology for evaluating knowledge in a specific context for a large number of students. The problem with MCQs is that students apply probabilistic approach to guess answers for scoring marks. Depending on the accuracy of the prediction, it may happen that the examinees who have little or no knowledge score more than or equal to the examinees who have better knowledge. Unexpected failures of the candidates who have passed MCQ based prestigious competitive examinations have also been observed. To overcome these contradictions, this article proposes a novel Confidence‐Based Evaluation Approach for MCQs (C‐BEAM) that curb the use of probabilistic prediction. The proposed marking scheme has been compared with well‐known marking schemes like no‐negative marking, one‐fourth negative marking, and one‐third negative marking. These marking schemes have been applied for the evaluation of eight MCQ quizzes on three engineering subjects that were attempted by more than 500 students at MNNIT Allahabad, India. The analysis of results proves C‐BEAM marking scheme as more effective than other compared methods in minimizing the effect of guessed answers. In addition to the effectiveness, C‐BEAM forms the basis for the confidence‐based feedback of students. This article also applies association rule mining methods to the evaluation data generated from C‐BEAM method for discovering association rules. These rules are confidence‐based feedback of students that would help instructors to improve the teaching process.

show abstract

“…The evaluation process of MCQs have been dealt by many researchers [10,11,[13][14][15]24,25,27]. Karen Scouller in ref.…”

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

C‐BEAM: A confidence‐based evaluation of MCQs for providing feedback to instructors

Aleem

Gore

2018

Comp Applic In Engineering

View full text Add to dashboard Cite

show abstract

“…As opposed to the free answer questions that are frequently used in LSA research (see, e.g., Diaz, Rifqi, Bouchon-Meunier, Jhean-Larose, & Denhière, 2008;Graesser, Wiemer-Hastings, Wiemer-Hastings, Kreuz, & Tutoring Research Group, 1999), this article addresses the question of how to automatically find the right answers to multiple choice questions, using LSA. An answer to this question could be interesting both from a cognitive point of view and in practical applications.…”

Section: B Semantic Spaces: To What Extent Are They Semantic?mentioning

confidence: 99%

Effect of tuned parameters on an LSA multiple choice questions answering model

Lifchitz

Jhean-Larose

Denhière

2009

Behavior Research Methods

Self Cite

View full text Add to dashboard Cite

This article presents the current state of a work in progress, whose objective is to better understand the effects of factors that significantly influence the performance of latent semantic analysis (LSA). A difficult task, which consisted of answering (French) biology multiple choice questions, was used to test the semantic properties of the truncated singular space and to study the relative influence of the main parameters. A dedicated software was designed to fine-tune the LSA semantic space for the multiple choice questions task. With optimal parameters, the performances of our simple model were quite surprisingly equal or superior to those of seventh-and eighthgrade students. This indicates that semantic spaces were quite good despite their low dimensions and the small sizes of the training data sets. In addition, we present an original entropy global weighting of the answers' terms for each of the multiple choice questions, which was necessary to achieve the model's success.

show abstract

“…Some works have focused on questionnaires allowing probabilistic [2,13,7,9] or fuzzy [12] answers, but very few allow belief answers. Diaz et al [5] have proposed a questionnaire to directly build a mass function, but it remains very unintuitive for any user. Other works [1,3] have considered directly generated mass functions without worrying about their construction.…”

Section: Introductionmentioning

confidence: 99%

Validation of Smets’ Hypothesis in the Crowdsourcing Environment

Thierry

Martin

Dubois

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In the late 1990s, Philippe Smets hypothesizes that the more imprecise humans are, the more certain they are. The modeling of human responses by belief functions has been little discussed. In this context, it is essential to validate the hypothesis of Ph. Smets. This paper focuses on the experimental validation of this hypothesis in the context of crowdsourcing. Crowdsourcing is the outsourcing of tasks to users of dedicated platforms. Two crowdsourcing campaigns have been carried out. For the first one, the user could be imprecise in his answer, for the second one he had to be precise. For both experiments, the user had to indicate his certainty in his answer. The results show that by being imprecise, users are more certain of their answers.

show abstract

Imperfect Answers in Multiple Choice Questionnaires

Cited by 9 publications

References 8 publications

C‐BEAM: A confidence‐based evaluation of MCQs for providing feedback to instructors

C‐BEAM: A confidence‐based evaluation of MCQs for providing feedback to instructors

Effect of tuned parameters on an LSA multiple choice questions answering model

Validation of Smets’ Hypothesis in the Crowdsourcing Environment

Contact Info

Product

Resources

About