The impact of a faculty development seminar on the quality of multiple-choice questions

Nemec, Eric C.; Welch, Beth

doi:10.1016/j.cptl.2015.12.008

Cited by 12 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reviewers may be fellow peers in the cohort, learners from other cohorts or disciplines or even senior/qualified colleagues such as post‐graduate trainees keen to advance their own involvement in education. Rubrics can guide the review processes to critique the item for issues such as factual accuracy, syntax and structure issues or flaws that assist the test‐wise learner 14 . Reviewer comments can be fed back to the original item author to allow them to make revisions before further editing and resubmission.…”

Section: Participant and Faculty Trainingmentioning

confidence: 99%

See 1 more Smart Citation

Playing the game: The educational role of gamified peer‐led assessment

Thampy

Walsh

Harris

2023

The Clinical Teacher

View full text Add to dashboard Cite

Section: Participant and Faculty Trainingmentioning

confidence: 99%

“…Rubrics can guide the review processes to critique the item for issues such as factual accuracy, syntax and structure issues or flaws that assist the test-wise learner. 14 Reviewer comments can be fed back to the original item author to allow them to make revisions before further editing and resubmission.…”

Section: Participant and Faculty Trainingmentioning

confidence: 99%

Playing the game: The educational role of gamified peer‐led assessment

Thampy

Walsh

Harris

2023

The Clinical Teacher

View full text Add to dashboard Cite

“…Item writers substantially influence the processes and outcomes involved in the development of assessments. Previous research demonstrates that SMEs who receive training in item writing produce higher quality items than untrained SMEs (AlFaris et al, 2015;Jozefowicz et al, 2002;Nemec & Welch, 2016), producing items with fewer common flaws associated with MCQs (e.g., "all of the above" or "none of the above," unclear wording, heterogeneous answer choice lengths; Nemec & Welch, 2016). These flaws may impact item characteristics, resulting in lower item discrimination (Tarrant & Ware, 2008) and inappropriately easy or difficult items (Downing, 2002;Rush et al, 2016).…”

Section: Influence Of Item Writers On Item Qualitymentioning

confidence: 99%

Do Subject Matter Experts’ Judgments of Multiple‐Choice Format Suitability Predict Item Quality?

Berenbon

McHugh

2023

Educational Measurement

View full text Add to dashboard Cite

To assemble a high‐quality test, psychometricians rely on subject matter experts (SMEs) to write high‐quality items. However, SMEs are not typically given the opportunity to provide input on which content standards are most suitable for multiple‐choice questions (MCQs). In the present study, we explored the relationship between perceived MCQ suitability for a given content standard and the associated item characteristics. Prior to item writing, we surveyed SMEs on MCQ suitability for each content standard. Following field testing, we then used SMEs’ average ratings for each content standard to predict item characteristics for the tests. We analyzed multilevel models predicting item difficulty (p value), discrimination, and nonfunctioning distractor presence. Items were nested within courses and content standards. There was a curvilinear relationship between SMEs’ ratings and item difficulty such that very low MCQ suitability ratings were predictive of easier items. After controlling for item difficulty, items with higher MCQ suitability ratings had higher discrimination and were less likely to have one or more nonfunctioning distractors. This research has practical implications for optimizing test blueprints. Additionally, psychometricians may use these ratings to better prepare for coaching SMEs during item writing.

show abstract

“…Some authors support that learning to compose MCQs is a "critical component to faculty development," and as such, the hiring institution along with the individual faculty are responsible for this training. 7,9 However, this development often comes as fragmented continuing education, and few nurse educators have formal preparation. 7 Because nursing is a practice-based profession, naturally, many nurse educators are clinically trained and then promoted "without receiving training on how to write good test questions."…”

mentioning

confidence: 99%

“…Unfortunately, these sources are also fraught with item writing flaws. [7][8][9] Furthermore, the time commitment required to develop MCQs should not be understated. It has been acknowledged by other researchers that although instructors spend considerable time planning lectures and course materials for students, insufficient time is allocated for test preparation and review before administration.…”

mentioning

confidence: 99%

Preparedness to Write Items for Nursing Education Examinations

et al. 2021

View full text Add to dashboard Cite

Background: Nursing programs in the United States primarily use multiple-choice questions to prepare students for the NCLEX. Currently, the optional next-generation NCLEX is testing the validity and reliability of situational case studies. Purpose: The purpose of this study was to investigate the preparedness and confidence of nurse educators in writing NCLEXstyle questions. Methods: A descriptive survey design with snowball sampling was used to collect the data and was sent to 1550 deans/ directors/coordinators of publicly available email addresses of accredited schools of nursing. Results: A total of 300 participants completed the survey from 44 states. When asked about confidence in writing NCLEX-style items, those who received formal education or training reported being somewhat confident (51%), whereas those who did not reported ambivalent or not confident (49%). Conclusion: Surveyed nurse educators lack confidence in writing questions and feel unprepared for the new question types instituted by the NCSBN in 2023. Nurse educators need continued education to develop and review NCLEX-style items and examinations.

show abstract

The impact of a faculty development seminar on the quality of multiple-choice questions

Cited by 12 publications

References 11 publications

Playing the game: The educational role of gamified peer‐led assessment

Playing the game: The educational role of gamified peer‐led assessment

Do Subject Matter Experts’ Judgments of Multiple‐Choice Format Suitability Predict Item Quality?

Preparedness to Write Items for Nursing Education Examinations

Contact Info

Product

Resources

About