I PALAVRAS-CHAVE

BACKGROUND: Progress tests are longitudinal assessments of students' knowledge based on successive tests. Calibration of the test difficulty is challenging, especially because of the tendency of item-writers to overestimate students' performance. The relationships between the levels of Bloom's taxonomy, the ability of test judges to predict the difficulty of test items and the real psychometric properties of test items have been insufficiently studied. OBJECTIVE: To investigate the psychometric properties of items according to their classification in Bloom's taxonomy and judges' estimates, through an adaptation of the Angoff method. DESIGN AND SETTING: Prospective observational study using secondary data from students' performance in a progress test applied to ten medical schools, mainly in the state of São Paulo, Brazil. METHODS: We compared the expected and real difficulty of items used in a progress test. The items were classified according to Bloom's taxonomy. Psychometric properties were assessed based on their taxonomy and fields of knowledge. RESULTS: There was a 54% match between the panel of experts' expectations and the real difficulty of items. Items that were expected to be easy had mean difficulty that was significantly lower than that of items that were expected to be medium (P < 0.05) or difficult (P < 0.01). Items with high-level taxonomy had higher discrimination indices than low-level items (P = 0.026). We did not find any significant differences between the fields in terms of difficulty and discrimination. CONCLUSIONS: Our study demonstrated that items with high-level taxonomy performed better in discrimination indices and that a panel of experts may develop coherent reasoning regarding the difficulty of items.

Objective: To validate the content of an educational material aimed at mothers and caregivers on healthy feeding for children less than two years of age.Methods: Quantitative study for content validation of an educational material containing three educational modules and respective folders, elaborated on the basis of official references for healthy feeding for children under two years old and adapted to the Brazilian context. Content validation was made through consensus conference in two stages by seven experts.Results: In the first stage, an individual and anonymous evaluation was made and the items analyzed by the seven experts averaged seven or more, with standard deviation below three. However, some items in the educational modules (operational and adherence) and Leaflet I (motivation and culture) reached cut-off values. The second stage involved a face-to-face meeting with five of the seven experts, expressing their opinions and seeking for a new consensus. The items whose results were close to the cut-off had an expressive increase in importance and consensus level after the meeting.Conclusions: The quantitative data obtained after the consensus meetings were superior to the predefined cut-offs, and the content of the proposed educational material was consensually validated by all participating experts. The consensus conference was an efficient methodological technique to build and validate educational instruments.

OBJECTIVES:The transition from face-to-face to remote teaching is yet to be fully understood. In clinical training, traditional teaching must prevail because it is essential for the acquisition of skills and professionalism. However, the responses of each school to the pandemic and the decision on when to resume clerkship rotations were mixed. In this study, we aimed to analyze whether the time to resume clerkship rotations was associated with the performance of the students by using a multi-institutional Progress Test. METHODS: This is a cross-sectional study conducted at nine different Brazilian medical schools that administer the same annual Progress Test for all students. We included information from 1,470 clerkship medical students and analyzed the time of clinical training interruption as the independent variable and the student's scores as the dependent variable. RESULTS: The comparisons of the students' scores between the schools showed that there are differences; however, they cannot be attributed to the time the clerkship rotations were paused. The correlation between the schools' average scores and the time to resume clerkship rotations was not significant for the fifth year (r= -0.298, p=0.436) and for the sixth year (r= -0.440, p=0.240). By using a cubic regression model, the time to resume clerkship rotations could explain 3.4% of the 5-year students' scores (p<0.001) and 0.9% of the 6year students, without statistical difference (p=0.085). CONCLUSIONS: The differences between the students' scores cannot be attributed to the time when the schools paused the clerkship rotations.