Use of a Committee Review Process to Improve the Quality of Course Examinations

Wallach, Paul M.; Crespo, Lynn M.; Holtzman, Kathleen Z.; Galbraith, Robert M.; Swanson, David B.

doi:10.1007/s10459-004-7515-8

Cited by 53 publications

(38 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The reliability for both sets of MCQs is nearly the same in this study. These findings call for the urgent need for proper construction of MCQs, adequate faculty training, and revision of MCQs by exam committees as well before the exam is due 22. There is no doubt that more distractors, if well designed, will add to the reliability of the test.…”

Section: Discussionmentioning

confidence: 99%

Comparison in the quality of distractors in three and four options type of multiple choice questions

Rahma

Shamad

Idris

et al. 2017

AMEP

View full text Add to dashboard Cite

IntroductionThe number of distractors needed for high quality multiple choice questions (MCQs) will be determined by many factors. These include firstly whether English language is their mother tongue or a foreign language; secondly whether the instructors who construct the questions are experts or not; thirdly the time spent on constructing the options is also an important factor. It has been observed by Tarrant et al that more time is often spent on constructing questions than on tailoring sound, reliable, and valid distractors.ObjectivesFirstly, to investigate the effects of reducing the number of options on psychometric properties of the item. Secondly, to determine the frequency of functioning distractors among three or four options in the MCQs examination of the dermatology course in University of Bahri, College of Medicine.Materials and methodsThis is an experimental study which was performed by means of a dermatology exam, MCQs type. Forty MCQs, with one correct answer for each question were constructed. Two sets of this exam paper were prepared: in the first one, four options were given, including one key answer and three distractors. In the second set, one of the three distractors was deleted randomly, and the sequence of the questions was kept in the same order. Any distracter chosen by less than 5% of the students was regarded as non-functioning. Kuder-Richardson Formula 20 (Kr-20) measures the internal consistency and reliability of an examination with an acceptable range 0.8–1.0. Chi square test was used to compare the distractors in the two exams.ResultsA significant difference was observed in discrimination and difficulty indexes for both sets of MCQs. More distractors were non-functional for set one (of four options), but slightly more reliable. The reliability (Kr-20) was slightly higher for set one (of four options). The average marks in option three and four were 34.163 and 33.140, respectively.ConclusionCompared to set 1 (four options), set 2 (of three options) was more discriminating and associated with low difficulty index but its reliability was low.

show abstract

Section: Discussionmentioning

confidence: 99%

Comparison in the quality of distractors in three and four options type of multiple choice questions

Rahma

Shamad

Idris

et al. 2017

AMEP

View full text Add to dashboard Cite

show abstract

“…Pre-established guidelines and an interdisciplinary review process results in improved item quality for in-house examinations (Wallach et al 2010). Once new items have been drafted, they must undergo a multistage, multiple-purpose review for content accuracy, editorial style, fairness and psychometric properties (Schmeiser and Welch 2006).…”

Section: Introductionmentioning

confidence: 99%

Faculty development on item writing substantially improves item quality

Naeem

Vleuten

AlFaris

2011

Adv in Health Sci Educ

View full text Add to dashboard Cite

The quality of items written for in-house examinations in medical schools remains a cause of concern. Several faculty development programs are aimed at improving faculty's item writing skills. The purpose of this study was to evaluate the effectiveness of a faculty development program in item development. An objective method was developed and used to assess improvement in faculty's competence to develop high quality test items. This was a quasi experimental study with a pretest-midtest-posttest design. A convenience sample of 51 faculty members participated. Structured checklists were used to assess the quality of test items at each phase of the study. Group scores were analyzed using repeated measures analysis of variance. The results showed a significant increase in participants' mean scores on Multiple Choice Questions, Short Answer Questions and Objective Structured Clinical Examination checklists from pretest to posttest (p < .0005). The effect sizes were 1.38, 3.84 and 2.20 for Multiple Choice Questions, Short Answer Questions and Objective Structured Clinical Examination, respectively. This study emphasizes that items written by faculty without faculty development are generally lacking in quality. It also provides evidence of the value of faculty development in improving the quality of items generated by faculty.

show abstract

“…Concerning question 3, 28 teachers believed that English language teachers need to be trained in quality assessment issues, 6 teachers 'were not sure', and only two teachers 'disagreed' with the statement. This is in accord with Wallach (2006) who believes that most of the teachers are often busy with teaching and they have little time dedicated to the construction and writing of exam items that would evaluate their students' true learning outcomes and provide future directions for them and their students (p, 62). For question 4, the vast majority of teachers (30) believed that quality assessment shapes quality teaching, 5 teachers 'were not sure', and only one teacher 'disagreed'.…”

Section: Resultsmentioning

confidence: 57%

“…The processes of pre-establishing exam item writing guidelines and establishing an exam review committee could promote quality assessment (Wallach, 2006). One of the reforms which the DELL has currently witnessed is that of forming an exam review committee.…”

Section: Forming An Exam Review Committeementioning

confidence: 99%

Towards Quality Assessment in an EFL Programme

Ali¹,

Ajmi²

2013

ELT

View full text Add to dashboard Cite

Assessment is central in education and the teaching-learning process. This study attempts to explore the perspectives and views about quality assessment among teachers of English as a Foreign Language (EFL), and to find ways of promoting quality assessment. Quantitative methodology was used to collect data. To answer the study questions, a questionnaire was distributed to 37 EFL teachers at a public college in Oman. The findings show that the majority of teachers are not satisfied with the current assessment practices and they believe that much effort is needed to promote quality assessment. Furthermore, the vast majority of teachers assume that using alternative assessment tools such as moderation, building an item bank, having tight exam policies and procedures, providing clear exam instructions, training teachers on quality assessment, and aligning assessment with teaching would help in fostering quality assessment. Based on teachers' views and findings from the literature, the study proposes a multidimensional model for quality assessment which may help in achieving and promoting quality assessment.

show abstract

Use of a Committee Review Process to Improve the Quality of Course Examinations

Abstract: Pre-established guidelines and an interdisciplinary review process resulted in improved item quality for in-house examinations.

Cited by 53 publications

References 9 publications

Comparison in the quality of distractors in three and four options type of multiple choice questions

Comparison in the quality of distractors in three and four options type of multiple choice questions

Faculty development on item writing substantially improves item quality

Towards Quality Assessment in an EFL Programme

Contact Info

Product

Resources

About