Reliability and Validity of the Chinese Version Appropriateness Evaluation Protocol

Abstract

ObjectiveTo adapt the Appropriateness Evaluation Protocol (AEP) to the specific settings of health care in China and to validate the Chinese version AEP (C-AEP).MethodsForward and backward translations were carried out to the original criteria. Twenty experts participated in the consultancy to form a preliminary version of the C-AEP. To ensure applicability, tests of reliability and validity were performed on 350 admissions and 3,226 hospital days of acute myocardial infraction patients and total hip replacement patients in two tertiary hospitals by two C-AEP reviewers and two physician reviewers. Overall agreement, specific agreement, and Cohen’s Kappa were calculated to compare the concordance of decisions between pairs of reviewers to test inter-rater reliability and convergent validity. The use of “overrides” and opinions of experts were recorded as measurements of content validity. Face validity was tested through collecting perspectives of nonprofessionals. Sensitivity, specificity, and predictive values were also reported.ResultsThere are 14 admission and 24 days of care criteria in the initial version of C-AEP. Kappa coefficients indicate substantial agreement between reviewers: with regard to inter-rater reliability, Kappa (κ) coefficients are 0.746 (95% confidence interval [CI] 0.644–0.834) and 0.743 (95% CI 0.698–0.767) of admission and hospital days, respectively; for convergent validity, the κ statistics are 0.678 (95% CI 0.567–0.778) and 0.691 (95% CI 0.644–0.717), respectively. Overrides account for less than 2% of all judgments. Content validity and face validity were confirmed by experts and nonprofessionals, respectively. According to the C-AEP reviewers, 18.3% of admissions and 28.5% of inpatient days were deemed inappropriate.ConclusionsThe C-AEP is a reliable and valid screening tool in China’s tertiary hospitals. The prevalence of inappropriateness is substantial in our research. To reduce inappropriate utilization, further investigation is needed to elucidate the reasons and risk factors for this inappropriateness.

Assessment of appropriateness of hospitalisations in Ukraine: analytical framework, method and findings

“…Like in the other countries,8 9 28–30 we have customised the European version of the AEP to adapt it to Ukrainian specifics. The scope of changes is marginal: we have amended some criteria, included relevant criteria and modified the wording.…”

“…Based on available international tools8 9 25–30 and existing approaches used in Ukraine,18–24 the tool for this study was developed and customised for the Ukrainian context in order to assess the appropriateness of hospitalisations and duration of inpatient stays. The tool included criteria for appropriateness of hospital admissions and a guide for expert evaluations of justified inpatient lengths of stay.…”

“…The group of reviewers controlled the adherence to the Delphi procedure. Criteria for hospital admissions were included in the customised tool based on an analysis of seven versions of the AEP: original US version (US-AEP),25 European (Е-AEP),26 Turkish (TR-AEP),29 Dutch (D-AEP),28 German (DE-AEP),30 Chinese (C-AEP)9 and Iranian version of the AEP (IR-AEP) 8. The customised Ukrainian AEP tool defined Ukraine-specific indications for hospital admission and criteria for the justification of inpatient length of stay—summarised in the Ukrainian Appropriateness Evaluation Protocol (UA-AEP) (see online supplementary file 1).…”

“…Overuse of hospital care is typical for many countries around the world, ranging from 1% to 54% in general,5 and up to 90% for some diseases 7. Unnecessary hospitalisations are a matter of concern for many nations, specifically due to the rising trend of healthcare expenditures and a significant proportion of hospital expenditures in overall health spending 8–12…”

Inappropriate admissions of the cardiology and orthopedics departments of a tertiary hospital in Shanghai, China

“…The admission section of the Chinese version of the Appropriateness Evaluation Protocol (C-AEP) was adopted as the screening tool in this study (see S1 Text) [18]. The original AEP was developed by Gertman and Restuccia in the U.S. in the 1980s [2], and was modified to accommodate local medical practices in many countries.…”