Experience of Service Questionnaire (ESQ) in children and adolescents: factor structure, reliability, validity, item parameters and interpretability of the parent version for practical use in Greece

Abstract

Background Health systems need tools to assess patient’s experience of service, but existing tools lack reliability and validity assessment. Our aim is to investigate the factor structure, reliability, validity, item parameters and interpretability of the parent version of the Experience of Service Questionnaire (ESQ) for practical use in Greece. Methods A total of 265 caregivers that were using mental health services in Greece participated in this study as part of the Nationwide cross-sectional survey from the Child and Adolescent Mental Health Initiative (CAMHI). Confirmatory Factor Analysis was used to test factor structure. Reliability of all models were measured with omega coefficients. Tobit regression analysis was used to test for convergent and discriminant validity with specifically designed questions. Item parameters were assessed via Item Response Theory. Interpretability was assessed by means of IRT-based scores. Results We found that ESQ is best represented and scored as a unidimensional construct, given potential subscales would not have enough reliability apart from a general factor. Convergent and discriminant validity was demonstrated, as caregivers who perceived that their child benefited from the received mental health care had 6.50 higher summed scores (SMD = 1.14, p < 0.001); while those who believed that their child needed additional help had 5.08 lower summed scores on the ESQ (SMD = −0.89, p < 0.001). Average z-scores provided five meaningful categories of services, in terms of user satisfaction, compared to the national average. Conclusions Our study presents evidence for the reliability and validity of the ESQ and provides recommendations for its practical use in Greece. ESQ can be used to measure experience of service and might help drive improvements in service delivery in the Greek mental health sector.