BackgroundMost studies on academic cheating rely on self-reported questionnaires and focus on the individual, overlooking cheating as a group activity. The aim of this study is to estimate the true prevalence of cheating/anomalies among medical students using a statistical index developed for this purpose, and to explore the existence of social networks between anomalies in students’ results.MethodsAngoff’s A index was applied to a sample of 30 written examinations, with a total of 1487 students and 7403 examinations taken, from the 2014/2015 academic year of the Faculty of Medicine of the University of Porto to detect anomaly pairs. All analyses are within the same academic year and not across years. Through simulations, the sensitivity and specificity of the statistical method was determined, and the true prevalence of anomalies/cheating was estimated. Networks of anomaly pairs were created to search for patterns and to calculate their density.ResultsThe percentage of students who cheated at least once increased with the year of medical school, being lowest in the first year (3.4%) and highest in the fifth (17.3%). The year of medical school was associated with anomalies (p < 0.05). The network’s density was also lowest in the first year (1.12E-04) and highest in the fifth (8.20E-04). The true prevalence of anomalies was estimated to be 1.85% (95%CI: 1.07–3.20%).ConclusionsThese findings suggest that some students are involved in social networks of cheating, which grow over time, resulting in an increase of anomalies/cheating in later academic years.Electronic supplementary materialThe online version of this article (10.1186/s12909-018-1299-7) contains supplementary material, which is available to authorized users.

BackgroundThe psychometric characteristics of multiple-choice questions (MCQ) changed when taking into account their anatomical sites and the presence of item-writing flaws (IWF). The aim is to understand the impact of the anatomical sites and the presence of IWF in the psychometric qualities of the MCQ.Results800 Clinical Anatomy MCQ from eight examinations were classified as standard or flawed items and according to one of the eight anatomical sites. An item was classified as flawed if it violated at least one of the principles of item writing. The difficulty and discrimination indices of each item were obtained. 55.8 % of the MCQ were flawed items. The anatomical site of the items explained 6.2 and 3.2 % of the difficulty and discrimination parameters and the IWF explained 2.8 and 0.8 %, respectively.ConclusionsThe impact of the IWF was heterogeneous, the Writing the Stem and Writing the Choices categories had a negative impact (higher difficulty and lower discrimination) while the other categories did not have any impact. The anatomical site effect was higher than IWF effect in the psychometric characteristics of the examination. When constructing MCQ, the focus should be in the topic/area of the items and only after in the presence of IWF.Electronic supplementary materialThe online version of this article (doi:10.1186/s13104-016-2202-4) contains supplementary material, which is available to authorized users.

In theory the formula scoring methods increase the reliability of multiple-choice tests in comparison with number-right scoring. This study aimed to evaluate the impact of the formula scoring method in clinical anatomy multiple-choice examinations, and to compare it with that from the number-right scoring method, hoping to achieve an evidence-based decision about test scoring rules. Two hundred and ninety-eight students completed an examination in clinical anatomy which included 40 multiple-choice questions with five response options each. Among these, 245 (82.2%) examinees were assessed according to the number-right scoring method (group A) while 53 (17.8%) were assessed according to the formula scoring method (group B). The prevalence of passing was significantly higher in group A than in group B, after correction of the pass and fail cutoffs for guessing (84.9% vs. 62.3%, P = 0.005), keeping a similar reliability in both groups (0.7 vs. 0.8, P = 0.094). Pearson Correlation coefficients between practical and theoretical examination scores were 0.66 [95%CI = (0.58-0.73)] and 0.72 [95%CI = (0.56-0.83)] for groups A and B, respectively. Based solely on the reliability and validity assessments, the test-maker could therefore use either scoring rules; however, if the test-maker also takes into account the students' ability to deduce answers with partial knowledge, then the number-right score rule is most appropriate.

Several studies have shown that the standard error of measurement (SEM) can be used as an additional "safety net" to reduce the frequency of false-positive or false-negative student grading classifications. Practical examinations in clinical anatomy are often used as diagnostic tests to admit students to course final examinations. The aim of this study was to explore the diagnostic value of SEM using the likelihood ratio (LR) in establishing decisions about students with practical examination scores at or below the pass/fail cutoff score in a clinical anatomy course. Two hundred sixty-seven students took three clinical anatomy practical examinations in 2011. The students were asked to identify 40 anatomical structures in images and prosected specimens in the practical examination. Practical examination scores were then divided according to the following cutoff scores: 2, 1 SEM below, and 0, 1, 2 SEM above the pass score. The positive predictive value (+PV) and LR of passing the final examination were estimated for each category to explore the diagnostic value of practical examination scores. The +PV (LR) in the six categories defined by the SEM was 39.1% (0.08), 70.0% (0.30), 88.9% (1.04), 91.7% (1.43), 95.8% (3.00), and 97.8% (5.74), respectively. The LR of categories 2 SEM above/below the pass score generated a moderate/large shift in the pre- to post-test probability of passing. The LR increased the usefulness and practical value of SEM by improving confidence in decisions about the progress of students with borderline scores 2 SEM above/below the pass score in practical examinations in clinical anatomy courses.

Background Computer and paper examinations in our days are constructed from an item pool which is regularly updated. Given the way that exams are created, one of the major concerns is the security of the items that are being used in order to ensure a good estimation of abilities. The aim of this study is to measure the prevalence of item pre-knowledge in our medical school. Methods The Deterministic, Gated Item Response Theory Model (DGM) was applied to estimate the prevalence of students who have had item pre-knowledge from six multiple choice examinations of the Clinical Anatomy course at the Faculty of Medicine of University of Porto. Each examination consisted of 100 items with an average of 200 students and 20% repeated items per examination. The estimation of the sensitivity and specificity was based on a simulation study. The sensitivity and specificity estimates, and apparent prevalence were used to estimate true prevalence of cheating students in the examinations under study. Results The specificity in the DGM for different simulation scenarios was between 68 and 98%, while the sensitivity ranged from 60 to 91%. The apparent prevalence was between 0.0 and 3.4%, while the true prevalence ranged from 1.2 to 3.7%. Conclusions The true prevalence was much lower compared to the students self-reported copying of responses from other students; however, it is important to keep monitoring the pre-knowledge prevalence in order to enforce measures in case an increase occurs. Electronic supplementary material The online version of this article (10.1186/s12909-019-1710-z) contains supplementary material, which is available to authorized users.