A short questionnaire to assess pediatric resident’s competencies: the validation process

Dalt, Liviana Da; Anselmi, Pasquale; Bressan, Silvia; Carraro, Silvia; Baraldi, Eugenio; Robusto, Egidio; Perilongo, Giorgio

doi:10.1186/1824-7288-39-41

Cited by 14 publications

(14 citation statements)

References 25 publications

(18 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This evaluation system is based on four validated questionnaires: the “Residents Assessment Questionnaire (AQ)”, the “Tutor-AQ”, the “Rotation-AQ” and the “Resident Affairs Committee-AQ” [ 2 , 3 ] that consist of various items evaluated on a four-point rating scale (from 1 “poor” to 4 “excellent”). The “Resident-AQs” are completed by the Division Chiefs, who, as mentioned above, are also the “tutors” that the residents evaluate by completing the “Tutor-AQ” questionnaire.…”

Section: Methodsmentioning

confidence: 99%

An evaluation system for postgraduate pediatric residency programs: report of a 3-year experience

et al. 2017

Self Cite

View full text Add to dashboard Cite

The way a postgraduate medical training program is organized and the capacity of faculty members to function as tutors and to organize effective professional experiences are among the elements that affect the quality of training. An evaluation system designed to target these elements has been implemented within the framework of the Pediatric Residency Program of the University of Padua (Italy). The aim of this report is to describe some aspects of the experience gained in the first 3 years of implementation of the system (2013–2015). Data were collected using four validated questionnaires: the “Resident Assessment Questionnaire”, the “Tutor-Assessment Questionnaire”, the “Rotation-Assessment Questionnaire”, and the “Resident Affairs Committee-Assessment Questionnaire”. The response rate was 72% for the “Resident Assessment Questionnaires”; 78% for the “Tutor-/Rotation-Assessment Questionnaires” and 84% for the “Resident Affair Committee-Assessment Questionnaires”. The scores collected were validated by psychometric tests. Conclusion: The high rates of completed questionnaires returned and the psychometric validation of the results collected indicate that the evaluation system reported herein can be effectively implemented. Efforts should be made to refine this system and, more importantly, to document its impact in improving the Pediatric Residency Program. What is known: • The elements that influence the quality of postgraduate training programs and the knowledge, performance, and competences of residents must be regularly assessed. • Comprehensive evaluation systems for postgraduate residency programs are not universally implemented also because quite often common guidelines and rules, well-equipped infrastructures, and financial resources are missing. What is new: • We show the feasibility of implementing an evaluation system that targets some of the key elements of a postgraduate medical training program in Italy, a European country in which the regulations governing training programs and, notably, the evaluation of residents are still being developed.

show abstract

Section: Methodsmentioning

confidence: 99%

An evaluation system for postgraduate pediatric residency programs: report of a 3-year experience

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…The validation of each of the three questionnaires has been conducted by taking into account aspects concerning the functioning of the items and that of the response scale, the dimensionality, reliability and construct validity of the questionnaire [ 11 ].…”

Section: Methodsmentioning

confidence: 99%

“…The Paediatric Residency Program (PRP) of the University of Padua, in Italy, while waiting for having the proxy-evaluation system implemented, addressed the issue of monitoring and evaluating the quality of the training by having the program going through a periodic, systematic ISO:9001 certification process [ 10 ]. Furthermore, it implemented a comprehensive evaluation system [ 11 ]. This system targets the following components: i) the residents’ knowledge and performance; ii) the quality of the teaching provided by the Faculty, iii) the quality of professional experience maturated during each rotation they go though and iv) of the functioning of the Resident Affair Committee (RAC), the body in charge of running the program.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Validating a set of tools designed to assess the perceived quality of training of pediatric residency programs

et al. 2015

Self Cite

View full text Add to dashboard Cite

BackgroundThe Paediatric Residency Program (PRP) of Padua, Italy, developed a set of questionnaires to assess the quality of the training provided by each faculty member, the quality of the professional experience the residents experienced during the various rotations and the functioning of the Resident Affair Committee (RAC), named respectively: “Tutor Assessment Questionnaire” (TAQ), “Rotation Assessment Questionnaire” (RAQ), and RAC Assessment Questionnaire”. The process that brought to their validation are herein presented.MethodBetween July 2012 and July 2013, 51 residents evaluated 26 tutors through the TAQ, and 25 rotations through the RAQ. Forty-eight residents filled the RAC Assessment Questionnaire. The three questionnaires were validated through a many-facet Rasch measurement analysis.ResultsIn their final form, the questionnaires produced measures that were valid, reliable, unidimensional, and free from gender biases. TAQ and RAQ distinguished tutors and rotations into 5–6 levels of different quality and effectiveness. The three questionnaires allowed the identification of strengths and weaknesses of tutors, rotations, and RAC. The agreement observed among judges was coherent to the predicted values, suggesting that no particular training is required for developing a shared interpretation of the items.ConclusionsThe work herein presented serves to enrich the armamentarium of tools that resident medical programs can use to monitor their functioning. A larger application of these tools will serve to consolidate and refine further the results presented.Electronic supplementary materialThe online version of this article (doi:10.1186/s13052-014-0106-2) contains supplementary material, which is available to authorized users.

show abstract

“…The higher the health of a person relative to the severity of an item, the higher the probability that that person will give to the item a response that is indicative of health (e.g., a response "no" to an item like "I have trouble falling asleep"). Because of their general applicability, Rasch models have been used in several areas, including personality and health assessment, education, and market research (see, e.g., Bechtel, 1985;Vidotto et al, 1998Vidotto et al, , 2006Vidotto et al, , 2007Vidotto et al, , 2010aDuncan et al, 2003;Cole et al, 2004;Bezruczko, 2005;Pallant and Tennant, 2007;Shea et al, 2009;Anselmi et al, 2011Anselmi et al, , 2013aAnselmi et al, ,b, 2015Da Dalt et al, 2013, 2017Balsamo et al, 2014;Rossi Ferrario et al, 2019;Sotgiu et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

New Perspectives in Computing the Point of Subjective Equality Using Rasch Models

2019

Self Cite

View full text Add to dashboard Cite

In psychophysics, the point of subject equality (PSE) is any of the points along a stimulus dimension at which a variable stimulus (visual, tactile, auditory, and so on) is judged by an observer to be equal to a standard stimulus. Rasch models have been found to offer a valid solution for computing the PSE when the method of constant stimuli is applied in the version of the method of transitions. The present work provides an overview of the procedures for computing the PSE using Rasch models and proposes some new developments. An adaptive procedure is described that allows for estimating the PSE of an observer without presenting him/her with all stimuli pairs. This procedure can be particularly useful in those situations in which psychophysical conditions of the individuals require that the number of trials is limited. Moreover, it allows for saving time that can be used to scrutinize the results of the experiment or to run other experiments. Also, the possibility of using Rasch-based fit statistics for identifying observers who gave unexpected judgments is explored. They could be individuals who, instead of carefully evaluating the presented stimuli pairs, gave random, inattentive, or careless responses, or gave the same response to many consecutive stimuli pairs. Otherwise, they could be atypical and clinically relevant individuals who deserve further investigation. The aforementioned developments are implemented using procedures and statistics that are well established in the framework of Rasch models. In particular, computerized adaptive testing procedures are used for efficiently estimating the PSE of the observers, whereas infit and outfit mean-squares statistics are used for detecting observers who gave unexpected judgments. Results of the analyses carried out on simulated data sets suggest that the proposed developments can be used in psychophysical experiments.

show abstract

A short questionnaire to assess pediatric resident’s competencies: the validation process

Cited by 14 publications

References 25 publications

An evaluation system for postgraduate pediatric residency programs: report of a 3-year experience

An evaluation system for postgraduate pediatric residency programs: report of a 3-year experience

Validating a set of tools designed to assess the perceived quality of training of pediatric residency programs

New Perspectives in Computing the Point of Subjective Equality Using Rasch Models

Contact Info

Product

Resources

About