Development of a shared item repository for progress testing in veterinary education

Abstract

IntroductionProgress testing in education is an assessment principle for the measurement of students’ progress over time, e.g., from start to graduation. Progress testing offers valid longitudinal formative measurement of the growth in the cognitive skills of the individual students within the subjects of the test as well as a tool for educators to monitor potential educational gaps and mismatches within the curriculum in relation to the basic veterinary learning outcomes.MethodsSix veterinary educational establishments in Denmark, Finland, Germany (Hannover), the Netherlands, Norway, and Sweden established in cooperation with the European Association of Establishments for Veterinary Education (EAEVE) a common veterinary item repository that can be used for progress testing in European Veterinary Education Establishments (VEEs), linear as well as computer adaptive, covering the EAEVE veterinary subjects and theoretical “Day One Competencies.” First, a blueprint was created, suitable item formats were identified, and a quality assurance process for reviewing and approving items was established. The items were trialed to create a database of validated and calibrated items, and the responses were subsequently psychometrically analyzed according to Modern Test Theory.ResultsIn total, 1,836 items were submitted of which 1,342 were approved by the reviewers for trial testing. 1,119 students from all study years and all partners VEEs participated in one or more of six item trials, and 1,948 responses were collected. Responses were analyzed using Rasch Modeling (analysis of item-fit, differential item function, item-response characteristics). A total of 821 calibrated items of various difficulty levels matching the veterinary students’ abilities and covering the veterinary knowledge domains have been banked.DiscussionThe item bank is now ready to be used for formative progress testing in European veterinary education. This paper presents and discusses possible pitfalls, problems, and solutions when establishing an international veterinary progress test.

Knowledge attainment, learning approaches, and self-perceived study burnout among European veterinary students

“…As a part of a progress test development project, “VetRepos,” between European establishments of veterinary education ( 4 , 24 ), veterinary students from seven European universities were invited to participate in a trial test (hereafter, the VetRepos test) that was open from June to August 14, 2023. The test comprised questions across the veterinary curriculum as outlined by EAEVE ( 5 ).…”

“…The data comprised responses to single-best-answer multiple-choice questions and true/false type statements based on additional questions or vignettes ( 4 ). The VetRepos items test knowledge across the entire veterinary curriculum, covering a range of disciplines in four subscales (17 disciplines in the basic sciences, 10 in companion animal and equine clinical sciences, 12 in animal production and production animal clinical sciences, and five in food safety and quality, public health, and one health concept).…”

“…Rasch model analysis ( 6 ) software RUMM ( 27 ) was used to place the student ability score (θ) and the item difficulty on a single linear logit scale where the mean item difficulty has a value of zero. The range of item difficulty for the items in the VetRepos database covered approximately six logit units ( 4 ).…”

“…After a pilot project at Utrecht University ( 2 ), PT has been implemented to our knowledge only in German-speaking programs in Germany, Austria, and Switzerland ( 3 ). In 2019, representatives from veterinary educational establishments in Denmark, Finland, Germany, the Netherlands, Norway, and Sweden, together with the European Association of Establishments for Veterinary Education (EAEVE), formed the VetRepos consortium for creating and calibrating an item repository for progress testing ( 4 ). The repository contains 821 test items of varying degrees of difficulty that test the veterinary curriculum, as outlined by EAEVE ( 5 ).…”

“…The item difficulty parameters from the VetRepos test have been calibrated using anchor items to compare the scores across test trials. A thorough description of the VetRepos test can be found in Schaper et al ( 4 ). A single test score per student was used in the current cross-sectional study.…”