Using a hybrid of artificial intelligence and template-based method in automatic item generation to create multiple-choice questions in medical education: Hybrid AIG

Abstract

ObjectivesTemplate-based automatic item generation (AIG) is more efficient than traditional item writing but it still heavily relies on expert effort in model development. While non-template-based AIG, leveraging artificial intelligence (AI), offers efficiency, it faces accuracy challenges. We aimed to integrate these approaches for leading to a significant rise in efficiency for AIG without sacrificing accuracy.Material and MethodsWe proposed the Hybrid AIG method that utilizes AI to generate item models (templates) and cognitive models to combine the advantages of the two AIG approaches. The Hybrid AIG consists of seven steps. The first five steps are carried out by an expert in a customized AI environment. Following a final expert review (Step 6), the content in the template can be used for item generation through a traditional (non-AI) software (Step 7). We used two multiple-choice questions for demonstration.ResultsWe demonstrated that AI is capable of generating item models and cognitive models for AIG under the guidance of a human expert. Leveraging AI in template development has substantially reduced the time investment from five hours to less than 10 minutes, and made it significantly less challenging.ConclusionsThe Hybrid AIG method transcends the traditional template-based approach by marrying the “art” that comes from AI as a “black box” with the “science” of algorithmic generation under the oversight of expert as a “marriage registrar”. It does not only capitalize on the strengths of both approaches but also mitigates their weaknesses, offering a human-AI collaboration to increase efficiency in medical education.