Methods for Improving Response Rates in an Emergency Department Setting – A Randomized Feasibility Study

Abstract

Standard single survey mode approaches, such as mail-only or telephone-only, generally produce lower response rates than sequential mixed mode (SMM) approaches. SMM approaches may be particularly important when lower response propensities and/or poor contact information exist, as with emergency department (ED) patients. Results were examined from a randomized feasibility study testing five survey administration modes among ED patients who are discharged-to-community. A total of 4,017 discharged patients from eight EDs during February 2016 were randomized in equal numbers to five modes: on-site distribution of a paper survey; mail notification of a web survey; email notification of a web survey; two-stage SMM (mailed survey with telephone follow-up); and three-stage SMM (email notification of a web survey with mail follow-up and then telephone follow-up). Sampled patients received a 43-item instrument focusing on ED patient experience. The two-stage SMM, commonly used for other patient experience surveys, was the reference mode. Three-stage SMM had a higher response rate (30.7%) than two-stage SMM (25.3%); the other three experimental modes had significantly lower response rates (0.8-9.6%; p<0.001 for all comparisons). On-site distribution was difficult to operationalize, possibly promoting selection bias. Valid email capture rates ranged from 4.0% to 48.3% by hospital and 30.1% overall, even though participating hospitals reported capture rates of at least 20%; the odds of having a valid email address were significantly lower for older and male patients. Findings highlight the limitations of web-only and on-site approaches, and identify SMM with a web-based component as a promising approach for survey administration in the ED setting. introduction Survey response rates are generally declining, particularly for surveys of young adults (Sheldon et al. 2007) and in settings with poor contact information (Brick and Williams 2012). The emergency department (ED) setting has both features, making this patient population particularly challenging to survey.

Development and validation of a patient experience of care survey for emergency departments

“…Four large‐scale experiments conducted in 2014, 2016 (two experiments), and 2018, tested the survey among patients from 12 hospitals, 50 hospitals, eight hospitals, and 16 hospitals, respectively. The primary aims of these experiments were to test mode protocols for survey administration and to test performance of survey items and modes of survey implementation; results are described elsewhere 12–15 . Item performance results informed the refinement of the survey instrument; item performance results from the 2018 experiment are described here.…”

“…The primary aims of these experiments were to test mode protocols for survey administration and to test performance of survey items and modes of survey implementation; results are described elsewhere. [12][13][14][15] Item performance results informed the refinement of the survey instrument; item performance results from the 2018 experiment are described here. Mode protocol results informed CMS' recommendation that the EDPEC DTC survey is administered in one of three modes: (1) standard mixed mode (mail with telephone follow-up), (2) web survey (by email invitation) with telephone follow-up, and (3) web survey (by email invitation) followed by a mailed survey with telephone follow-up.…”

A Systematic Review of Strategies to Enhance Response Rates and Representativeness of Patient Experience Surveys

“…The overall response rate to the on-site survey was very low, but exceeded that of a web-based survey (10% for on-site vs. 5% web survey). 42 Another study tested the use of tablets to administer in-person surveys to parents of hospitalized children while they were waiting to be discharged, and found a substantially higher response rate for this protocol than to mail administration (71% tablet administration vs. 16% mail). 40 Tablet respondents were significantly more likely to be fathers, more likely to have a high school education or less, less likely to be White, and more likely to be publicly insured than mail respondents.…”

Effects of Push-To-Web Mixed Mode Approaches on Survey Response Rates: Evidence from a Randomized Experiment in Emergency Departments

“…In this paper, we examine a patient experience survey of hospital-based emergency department (ED) patients discharged to home after their ED visit (rather than admitted to the hospital), referred to as discharged-to-community (Parast et al 2019) patients. As previous studies have found very low survey response rates among such patients, less than 20% (Mathews et al 2019;Weinick et al 2014), we sought to understand how response rates could be improved in this population.…”

“…Previous work in the ED DTC population has shown that single-mode protocols, such as mail-only or web-only, and distribution within the ED do not perform well (1-14% response rate), but sequential mixed-mode approaches may have promise (Mathews et al 2019;Parast et al 2019). In this paper, we describe results from a randomized test of nine experimental survey administration protocols, eight of which began with an invitation to a web survey, then followed by a mailed survey, a telephone survey or both.…”