Evaluating the impact of faculty development programme initiative: are we really improving skills in MCQ writing?

Kiran, Faiza; Ayub, Rukhsana; Rauf, Ayesha; Qamar, Khadija

doi:10.47391/jpma.1207

Cited by 6 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Faculty members expressed appreciation for experiential learning facilitated by educationalists during the process. This aligns with the findings of Kiran 16 and Notzer 17 , who emphasized the transformational nature of learning during hands-on practice and feedback in skill enhancement.…”

Section: Discussionsupporting

confidence: 89%

Exploring the experiences of content experts with item vetting during item bank development

Aziz,

Sabqat,

Kiran

et al. 2024

Pak J Med Sci

View full text Add to dashboard Cite

Objective: To explore content experts’ experiences with item vetting during item bank development at a public sector medical university of Rawalpindi, Pakistan. Methods: An exploratory study was carried out from December 2022 to February 2023 at a public sector medical college of Rawalpindi. A purposive sampling technique was employed to collect data from all content experts of the study institute who participated in item vetting activity during pre-exam moderation in the university. A pilot-tested semi-structured interview guide was utilized, interviews were audio recorded and later transcribed. Participants’ anonymity was ensured. Various quality assurance strategies were employed to ensure the trustworthiness of the findings. Thematic analysis was performed on the transcribed data and themes were finalized by achieving consensus among all authors. Results: Six themes overarching the fourteen subthemes emerged from the data. Participants expressed a profound sense of satisfaction and valued their experience in refining expertise in constructing multiple-choice questions (MCQs). It was widely acknowledged that such activities not only contribute to the enhancement of item development skills but also improve quality of items. Conclusions: The consistent implementation of item vetting routines, in conjunction with diligent adherence to item writing protocols, contributes to quality assurance measures in assessment. Item bank development for fair and transparent assessment ensures production of competent healthcare professionals filtering incompetent ones hence improving health care services in the community. doi: https://doi.org/10.12669/pjms.40.6.8664 How to cite this: Aziz A, Sabqat M, Kiran F, Mirza TI. Exploring the experiences of content experts with item vetting during item bank development. Pak J Med Sci. 2024;40(6):1241-1246. doi: https://doi.org/10.12669/pjms.40.6.8664 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

show abstract

Section: Discussionsupporting

confidence: 89%

Exploring the experiences of content experts with item vetting during item bank development

Aziz,

Sabqat,

Kiran

et al. 2024

Pak J Med Sci

View full text Add to dashboard Cite

show abstract

“…Our study adds to studies which approved the importance of peer review in the improvement of the quality of MCQs (15,16,17).…”

Section: Resultsmentioning

confidence: 80%

Did faculty training on item construction and peer review improve the quality of MCQs? An observational study in a medical school

Hassan

Alzahrani

Ahmad

et al. 2023

Preprint

View full text Add to dashboard Cite

Background: multiple choice questions (MCQs) test is the most popular assessment tool that used in medical schools, faculty training on construction of MCQs and peer review are the main factors that contributing to have a good quality MCQs test. This study aimed to evaluate the effect of these two factors on the quality of MCQs test. Methods: an extensive training program was conducted for the staff members who participated in teaching and assessment of undergraduate medical students of clinical biochemistry and psychiatric courses during the academic year 2020-2021. Post-test item analysis was used to compare pre-training with post-training item analysis parameters. The Chi-square test was applied, the level of significance was 5%. Results: The difficulty index showed great improvement in the number of items which lie in the acceptable range of difficulty from 16% and 25% before the training program, to 61% and 42% after the training program for psychiatry and clinical biochemistry courses respectively. This is significantly different (P<0.05). In psychiatry course the number of questions with negative discrimination power decreased from 36% before training program to 10% after it (P value >0.05). No significant improvement was noticed in biochemistry course regarding this aspect. The percentage of test items where all distracters were functioning was increased from 11% to 41% in psychiatry course (P value=0.0002), and from 12% to 46%, in biochemistry course (P value=0.0001). This is a significant improvement. Reliability after the training program and peer review processes also increased in both courses. However, this is not statistically significant. Conclusions: A dramatic improvement was observed in the quality of MCQs after peer review and training program. We recommend adopting training program on construction of MCQs to be mandatory for all medical school staff members in Saudi Arabia.

show abstract

“…They can encourage staff participation and involvement owing to their concise nature and convenience. However, shorter sessions must be organized more frequently to reinforce the principles and guidelines to be followed [ 24 ]. Moore’s Framework, Bloom’s taxonomy, and Miller’s pyramid can be used to further support teachers’ ability to construct high-quality MCQs [ 25 ].…”

Section: Discussionmentioning

confidence: 99%

Efficacy of Faculty Development Training Workshops (FDTWs) on Writing High-Quality Multiple-Choice Questions at Northern Border University (NBU) in the Kingdom of Saudi Arabia (KSA)

Agarwal,

Khattak,

Esmaeel

et al. 2024

Cureus

View full text Add to dashboard Cite

Background: A multiple-choice question (MCQ) is a frequently used assessment tool in medical education for both certification and competitive examinations. Well-constructed MCQs impact the utility of the assessment and, thus, the fate of the examinee. Aims and objectives: To analyze the basic science faculty perceptions of writing high-quality MCQs, to create awareness of item-writing flaws in constructing high-quality MCQs, and to determine the impact of faculty development training workshops (FDTWs) on MCQ writing skills. Material and methods: An online workshop was held over two weeks for basic science faculty to learn high-quality MCQ construction. Faculty-made MCQs were analyzed for flaws, and a questionnaire assessed the impact of the workshop on MCQ construction. Pre- and post-workshop responses were compared to evaluate the necessity of such workshops for improving faculty skills in MCQ assessments. Results: A total of 47 (83.2%) of participating faculty believed the workshop could reduce MCQ construction errors. The participants agreed that a series of workshops were needed for lasting improvements in MCQ construction. Conclusions: One-day short-duration workshops, such as the current one alone, cannot achieve the objectives of training participants to write high-quality MCQs. To improve student assessment through high-quality MCQs, the faculty needs to be exposed to continuous and frequent sessions that will help them.

show abstract

Evaluating the impact of faculty development programme initiative: are we really improving skills in MCQ writing?

Cited by 6 publications

References 0 publications

Exploring the experiences of content experts with item vetting during item bank development

Exploring the experiences of content experts with item vetting during item bank development

Did faculty training on item construction and peer review improve the quality of MCQs? An observational study in a medical school

Efficacy of Faculty Development Training Workshops (FDTWs) on Writing High-Quality Multiple-Choice Questions at Northern Border University (NBU) in the Kingdom of Saudi Arabia (KSA)

Contact Info

Product

Resources

About