Validity and reliability of a health care service evaluation instrument for tuberculosis

Abstract

OBJECTIVE To evaluate the validity and reliability of an instrument that evaluates the structure of primary health care units for the treatment of tuberculosis.METHODS This cross-sectional study used simple random sampling and evaluated 1,037 health care professionals from five Brazilian municipalities (Natal, state of Rio Grande do Norte; Cabedelo, state of Paraíba; Foz do Iguaçu, state of Parana; Sao José do Rio Preto, state of Sao Paulo, and Uberaba, state of Minas Gerais) in 2011. Structural indicators were identified and validated, considering different methods of organization of the health care system in the municipalities of different population sizes. Each structure represented the organization of health care services and contained the resources available for the execution of health care services: physical resources (equipment, consumables, and facilities); human resources (number and qualification); and resources for maintenance of the existing infrastructure and technology (deemed as the organization of health care services). The statistical analyses used in the validation process included reliability analysis, exploratory factor analysis, and confirmatory factor analysis.RESULTS The validation process indicated the retention of five factors, with 85.9% of the total variance explained, internal consistency between 0.6460 and 0.7802, and quality of fit of the confirmatory factor analysis of 0.995 using the goodness-of-fit index. The retained factors comprised five structural indicators: professionals involved in the care of tuberculosis patients, training, access to recording instruments, availability of supplies, and coordination of health care services with other levels of care. Availability of supplies had the best performance and the lowest coefficient of variation among the services evaluated. The indicators of assessment of human resources and coordination with other levels of care had satisfactory performance, but the latter showed the highest coefficient of variation. The performance of the indicators “training” and “access to recording instruments” was inferior to that of other indicators.CONCLUSIONS The instrument showed feasibility of application and potential to assess the structure of primary health care units for the treatment of tuberculosis.

Development and validation of a questionnaire to evaluate infection control in oral radiology

“…The psychometric indicators were analyzed based on reliability and validity. 14,17,18 Reliability was determined using test-retest and internal consistency. 13,15,17,19,24,25,31,32 In this stage, the questionnaire was applied and reapplied to the 115 students at 2 different times, with a 1-week interval between them.…”

“…13,15,17,19,24,25,31,32 In this stage, the questionnaire was applied and reapplied to the 115 students at 2 different times, with a 1-week interval between them. 33,34 To determine intraevaluator reproducibility, the ordinal variables were analyzed using the intraclass correlation coefficient (ICC) with a cut-off value .0.70, 17,18,21,31 and kappa was used for the yes/no questions with a cut-off value .0.60. 17,33 To determine internal consistency, Cronbach's alpha was used with a cut-off value .0.70.…”

“…17,33 To determine internal consistency, Cronbach's alpha was used with a cut-off value .0.70. 15,18,19,24,27,32,34 The construct validity of the questionnaire was determined using factor analysis and the knowngroups technique. 17,21 In this case, the questionnaire was also applied to professors of oral radiology and PhD students in oral radiology in addition to the 641 dentistry students, which allowed for a comparison between the scores of these group answers.…”

“…It must be stressed that, in order to deliver scientifically sound results with clinical and academic relevance, a questionnaire must be standardized and have good psychometric qualities 13 such as validity and reliability. [14][15][16][17][18] The extent to which one test is precise in terms of assessing what it is supposed to assess, i.e. its variable (or variables) of interest, is known as validity.…”

“…17,21 The former validates the construct by means of a statistical model, whereby the several items designed to assess one domain within a construct are supposed to be more highly correlated with one another than with other items measuring other domains in the same questionnaire. 15,17,18,22,23 The known-groups technique, on the other hand, tests one group in which the construct outcome is established against another group in which the outcome is not present, 15,17,21 thus determining the discriminatory power of the questionnaire.…”