E-assessment with multiple choice questions: a qualitative study of teachers' opinions and experience regarding the new assessment strategy

Babo, Rosalina; Suhonen, Jarkko

doi:10.1504/ijlt.2018.10017163

Cited by 2 publications

(18 citation statements)

References 28 publications

(149 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(a) its conditioning for automatic correction, which has contributed to the widespread use of this kind of assessment (Babo & Suhonen, 2018;Llamas-Nistal, Fernández-Iglesias, González-Tato, & Mikic-Fonte, 2013;Sorensen, 2013). "From the lecturers' point of view, this tool facilitates the classification and management of errors, guaranteeing the coherence of the grading and revision processes" (Llamas-Nistal et al, 2013, p. 86), and significantly reduces the burden associated with testing large student cohorts (Babo & Azevedo, 2013;Bull & McKenna, 2004, as cited in Nicol, 2007);…”

Section: Learning Management Systems (Lms) and Multiple-choice Questions (Mcq)mentioning

confidence: 99%

“…(a) the 'luck factor', making it possible for a failing student to pass due to randomly selected questions, allowing better marks without the corresponding knowledge (Babo & Suhonen, 2018;Llamas-Nistal et al, 2013;Maier et al, 2016;Sorensen, 2013);…”

Section: Learning Management Systems (Lms) and Multiple-choice Questions (Mcq)mentioning

confidence: 99%

“…(b) the matter of using question banks, which are accompanied by a set of problems, such as being time consuming, problems associated with the difficulty of developing good MCQs, the likelihood of security issues, and consequences associated with system instability (Babo & Suhonen, 2018;Ellaway & Masters, 2008;Ferrão, 2010;Miguel, Caballé, Xhafa, & Prieto, 2015;Sorensen, 2013);…”

Section: Learning Management Systems (Lms) and Multiple-choice Questions (Mcq)mentioning

confidence: 99%

“…(d) the absence of the ability to evaluate other types of skills/competencies (the so-called 'soft skills') in comparison to other modalities of assessment, like open-response, traditional on paper tests, or even directly on applications (such as Database Management systems (DBSM) or spreadsheets) (Babo & Suhonen, 2018;Llamas-Nistal et al, 2013).…”

Section: Learning Management Systems (Lms) and Multiple-choice Questions (Mcq)mentioning

confidence: 99%

“…The students' perceptions about the assessment affects their learning (Lynam & Cachia, 2018), therefore there was a need to understand the opinions of the people involved. This was the reason why Babo, Azevedo, and Suhonen (2015) and Babo and Suhonen (2018) have researched the opinions and perceptions of students and teachers, respectively, about eassessment with MCQ. However, the study from 2015 was limited to a group of 84 students of the school year 2012/2013.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

E- Assessment with Multiple-Choice Questions: A 5 Year Study of Students’ Opinions and Experience

et al. 2020

Self Cite

View full text Add to dashboard Cite

Aim/Purpose: The aim of this study is to understand student’s opinions and perceptions about e-assessment when the assessment process was changed from the traditional computer assisted method to a multiple-choice Moodle based method. Background: In order to implement continuous assessment to a large number of students, several shifts are necessary, which implies as many different tests as the number of shifts required. Consequently, it is difficult to ensure homogeneity through the different tests and a huge amount of grading time is needed. These problems related to the traditional assessment based on computer assisted tests, lead to a re-design of the assessment resulting in the use of multiple-choice Moodle tests. Methodology: A longitudinal, concurrent, mixed method study was implemented over a five-year period. A survey was developed and carried out by 815 undergraduate students who experienced the electronic multiple-choice questions (eMCQ) assessment in the courses of the IS department. Qualitative analyses included open-ended survey responses and interviews with repeating students in the first year. Contribution: This study provides a reflection tool on how to incorporate frequent moments of assessment in courses with a high number of students without overloading teachers with a huge workload. The research analysed the efficiency of assessing non-theoretical topics using eMCQ, while ensuring the homogeneity of assessment tests, which needs to be complemented with other assessment methods in order to assure that students develop and acquire the expected skills and competencies. Findings: The students involved in the study appreciate the online multiple-choice quiz assessment method and perceive it as fair but have a contradictory opinion regarding the preference of the assessment method, throughout the years. These changes in perception may be related to the improvement of the question bank and categorisation of questions according to difficulty level, which lead to the nullification of the ‘luck factor’. Other major findings are that although the online multiple-choice quizzes are used with success in the assessment of theoretical topics, the same is not in evidence regarding practical topics. Therefore, this assessment needs to be complemented with other methods in order to achieve the expected learning outcomes. Recommendations for Practitioners: In order to be able to evaluate the same expected learning outcomes in practical topics, particularly in technology and information systems subjects, the evaluator should complement the online multiple-choice quiz assessment with other approaches, such as a PBL method, homework assignments, and/or other tasks performed during the semester. Recommendation for Researchers: This study explores e-assessment with online multiple-choice quizzes in higher education. It provides a survey that can be applied in other institutions that are also using online multiple-choice quizzes to assess non-theorical topics. In order to better understand the students’ opinions on the development of skills and competencies with online multiple-choice quizzes and on the other hand with classical computer assisted assessment, it would be necessary to add questions concerning these aspects. It would then be interesting to compare the findings of this study with the results from other institutions. Impact on Society: The increasing number of students in higher education has led to a raised use of e-assessment activities, since it can provide a fast and efficient manner to assess a high number of students. Therefore, this research provides meaningful insight of the stakeholders’ perceptions of online multiple-choice quizzes about practical topics. Future Research: An interesting study, in the future, would be to obtain the opinions of a particular set of students on two tests, one of the tests using online multiple-choice quizzes and the other through a classical computer assisted assessment method. A natural extension of the present study is a comparative analysis regarding the grades obtained by students who performed one or another type of assessment (online multiple-choice quizzes vs. classical computer assisted assessment).

show abstract

Section: Learning Management Systems (Lms) and Multiple-choice Questions (Mcq)mentioning

confidence: 99%

Section: Learning Management Systems (Lms) and Multiple-choice Questions (Mcq)mentioning

confidence: 99%

Section: Learning Management Systems (Lms) and Multiple-choice Questions (Mcq)mentioning

confidence: 99%

Section: Learning Management Systems (Lms) and Multiple-choice Questions (Mcq)mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

E- Assessment with Multiple-Choice Questions: A 5 Year Study of Students’ Opinions and Experience

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

Improving Workgroup Assessment with WebAVALIA: The Concept, Framework and First Results

2020

View full text Add to dashboard Cite

Aim/Purpose: The purpose of this study is to develop an efficient methodology that can assist the evaluators in assessing a variable number of individuals that are working in groups and guarantee that the assessment is dependent on the group members’ performance and contribution to the work developed. Background: Collaborative work has been gaining more popularity in academic settings. However, group assessment needs to be performed according to each individual’s performance. The problem rests on the need to distinguish each member of the group in order to provide fair and unbiased assessments. Methodology: Design Science Research methodology supported the design of a framework able to provide the evaluator with the means to distinguish individuals in a workgroup and deliver fair results. Hevner’s DSR guidelines were fulfilled in order to describe WebAVALIA. To evaluate the framework, a quantitative study was performed and the first results are presented. Contribution: This paper provides a methodological solution regarding a fair evaluation of collaborative work through a tool that allows its users to perform their own assessment and peer assessment. These are made accordingly to the user’s perspectives on the performance of each group member throughout the work development. Findings: The first analysis of the results indicates that the developed method provides fairness in the assessment of group members, delivering a distinction amongst individuals. Therefore, each group member obtains a mark that corresponds to their specific contribution to the workgroup. Recommendations for Practitioners: For those who intend to apply this workgroup assessment method, it is relevant to raise student awareness about the methodology that is going to be used. That is, all the functionalities and steps in WebAVALIA have to be thoroughly explained before beginning of the project. Then, the evaluators have to decide about the students’ intermediate voting, namely if the evaluator chooses or not to publish student results throughout the project’s development. If there is the decision to display these intermediate results, the evaluator must try to encourage collaboration among workgroup members, instead of competition. Recommendation for Researchers: This study explores the design and development of an e-assessment tool – WebAVALIA. In order to assess its feasibility, its use in other institutions or contexts is recommended. The gathering of user opinions is suggested as well. It would then be interesting to compare the findings of this study with the results from other experimentations Impact on Society: Sometimes, people develop a rejection of collaborative work because they feel exploited due to the biased evaluation results. However, the group members assessment distinction, according to each one’s performance, may give each individual a sense of fairness and reward, leading to an openness/willingness towards collaborative work. Future Research: As future work, there are plans to implement the method in other group assessment contexts – such as sports and business environments, other higher education institutions, technical training students – in other cultures and countries. From this myriad of contexts, satisfaction results would be compared. Other future plans are to further explore the mathematical formulations and the respective WebAVALIA supporting algorithms.

show abstract

E-assessment with multiple choice questions: a qualitative study of teachers' opinions and experience regarding the new assessment strategy

Cited by 2 publications

References 28 publications

E- Assessment with Multiple-Choice Questions: A 5 Year Study of Students’ Opinions and Experience

E- Assessment with Multiple-Choice Questions: A 5 Year Study of Students’ Opinions and Experience

Improving Workgroup Assessment with WebAVALIA: The Concept, Framework and First Results

Contact Info

Product

Resources

About