Errors in Self-Reports of Health Services Use

Abstract

Background Most Alzheimer's disease clinical trials that compare the use of health services rely on reports of caregivers. The goal of this study was to assess the accuracy of self-reports among older adults with Alzheimer's disease and their caregiver proxy respondents. This issue is particularly relevance to Alzheimer's disease clinical trials because inaccuracy can lead both to loss of power and increased bias in study outcomes. Methods We compared respondent accuracy in reporting any use and in reporting the frequency of use with actual utilization data as documented in a comprehensive database. We next simulated the impact of under-reporting and over-reporting on sample size estimates and treatment effect bias for clinical trials comparing utilization between experimental groups. Results Respondents self-reports have a poor level of accuracy with Kappa values often below 0.5. Respondents tend to under-report use even for rare events such as hospitalizations and nursing home stays. In analyses simulating under- and over-reporting of varying magnitude, we found that errors in self-reports can increase the required sample size by 15–30%. In addition, bias in the reported treatment effect ranged from 3%–18% due to both under-reporting and over-reporting errors. Conclusions Use of self-report data in clinical trials of Alzheimer's disease treatments may inflate sample size needs. . Even when adequate power is achieved by increasing sample size, reporting errors can result in a biased estimate of the true effect size of the intervention.

Study partners should be required in preclinical Alzheimer’s disease trials

“…In preclinical AD, FDA guidance indicates that reducing decline on a single primary outcome measure may be sufficient to achieve approval [ 40 ], with post-approval studies to confirm that treatments result in clinically meaningful functional benefit. Though novel approaches to demonstrating clinical benefit should be pursued, such as assessing resource utilization through medical record or claims data [ 41 ], long-term extension studies using traditional global or functional outcome measures seem the most probable approach. These outcome measures require a study partner.…”

Accuracy of Caregiver Proxy Reports of Home Care Service Use

“…For all four services examined, especially notable for home support and home nursing, overreporting was more common than underreporting. Other studies have tended to find underreporting more common than overreporting (Bhandari & Wagner, 2005; Callahan et al, 2015; Ritter et al, 2001; Schmitz, Russell, & Cutrona, 2002), but they have not examined home care services, nor typically, care to those with dementia.…”

“…Chi-square tests of independence were used for categorical variables. The variables chosen were based on past research (Callahan et al, 2015) or intuitive relevance (caregiver: age, years caring, hours/week caring, burden, relation, and lives with care recipient; care recipient: age, age at diagnosis, gender, number of chronic conditions, and number of services.…”

“…Bhandari and Wagner’s (2005) review of 42 studies on the accuracy of self-reported utilization data noted “variable” accuracy and did not include studies on proxy reports by caregivers to those with dementia. Callahan and colleagues’ (2015) recent comparison of caregivers to those with dementia focused on clinical trials data. Furthermore, existing research focuses on hospital use, nursing home admissions, and clinical visits, not on home care service utilization.…”

Consider the Source

“…1 Their analyses of a subset of data from a randomized controlled trial of a coordinated care model, which successfully enrolled a real world sample of 173 patients with Alzheimer’s disease, 2 found poor agreement between informant reports and objective healthcare utilization data. Informants most often underreported resource utilization.…”