Using Item Response Theory to evaluate the psychometric characteristics of questions in a Brazilian programme and the performance of dental teams in primary care

Abstract

Objectives First, to assess the psychometric properties of key questions included in a public sector evaluation of primary dental care in Brazil; and second, to evaluate the performance of dental teams in relation to these items. Methods Secondary analysis of a national primary care dataset monitoring quality and access to dental care. Data were collected through face-to-face interviews with representatives of dental teams participating in the ‘National Programme for Improving Access and Quality of Primary Care’. Twenty-three mandatory questions about the dentists’ reported delivery of dental procedures were included in the analysis. Item Response Theory (IRT) modelling was applied to measure the psychometric properties of the instrument—level of difficulty and discrimination parameter of each item—and then to estimate dental team performance scores based on these parameters. Based on IRT, possible scores ranged from -4 to +4. Results Three of the 23 mandatory items were removed due to poor internal consistency, resulting in a scale of 20 items for assessing dental team performance. The results showed variation in procedures delivered by the dental teams; whilst more than a half of the procedures were executed by at least 80% of the dental teams, those relating to dentures (partial/total) and frenectomy (lingual/labial) were performed by less than 30%. Amongst the 20 items included in the model, those related to partial/total dentures and oral cancer follow-up presented higher levels of difficulty and were less frequently provided. The items relating to the treatment of deciduous teeth and access to the dental pulp of permanent teeth had the highest discrimination parameters and, consequently, greater weight in the performance’s score estimation; therefore, dental teams that did not perform these items had the lowest performance scores. In the present study, dental team performance scores ranged from -3.66 to +1.87 with a mean/median of -0.06/+0.01. Conclusion The findings suggest that whilst the items within the instrument demonstrated some potential to discriminate between poor and very poor teams, they were ineffective in discriminating between poor and good teams. Whilst Brazilian dental teams perform many mandatory procedures, variation in the nature of their delivery of care requires further investigation to enhance service provision to the population.

The Influence of Dentists’ Profile and Health Work Management in the Performance of Brazilian Dental Teams

“…Through the IRT, items' psychometric characteristics were estimated, and a one-off performance score was attributed to each dental team evaluated in the second cycle of PMAQ-AB. In the present study, dental team scores ranged from -3.66 to +1.87 (mean of -0.06; median of +0.01) [ 11 , 15 ].…”

“…In our analysis, the dependent variable was “dental team performance” obtained through dentist's report about the execution of 20 mandatory dental procedures of PHC dentistry and presented in full in our previous paper [ 11 ]. The questions included the execution of common dental procedures (preventive, restorative/prosthetic, surgical, and endodontic), besides the identification, referral, and monitoring of oral cancer cases.…”

“…Preliminary analysis of data from the second evaluation cycle of PMAQ-AB, occurred between 2013 and 2014, showed that it is necessary to expand the provision of dentures and dental prostheses [ 10 ]. The psychometric analyses of the questions related to the execution of the dental procedures attributed a score to the dental teams that we called “dental team performance,” varying between -3.66 and +1.87 (mean of -0.06; median of +0.01) and showed that these items were better in discriminating the dental teams with negative scores, especially those between -2 and -3 [ 11 ]. These preliminary analyses were necessary to achieve the aim of the present study, that is, to evaluate the influence of dentists' profile and health work management in the performance of primary care dental teams of the Brazilian National Health System.…”

Practice-based analysis of direct posterior dental restorations performed in a public health service: Retrospective long-term survival in Brazil

“…The ‘National Program for Improving Access to and Quality of Primary Care’ in Brazil evaluated the southeast region (5,027 dental teams) between 2013 and 2014 and showed that 98.4% of the dental teams performed resin composite restorations and 93.5% performed amalgam posterior restorations [ 25 , 26 ]. In the past 25 years, there has been a steady growth in the use of resin composite materials for the restoration of posterior teeth.…”

Performance Evaluation of Oral Health Teams in Brazil: An Item Response Theory Approach

“…Results evaluating the performance of Brazilian OHTs in the first cycle of PMAQ-AB showed that teams with high performance develop more actions to identify oral cancer and provide more dental prostheses in PHC [8]. Results of the second cycle also showed that the actions focused on oral cancer screening and the offer of dental prostheses were related to higher levels of difficulty and were conducted by OHTs with higher performance scores [9].…”