Does scrolling affect measurement equivalence of electronic patient-reported outcome measures (ePROM)? Results of a quantitative equivalence study

Abstract

Background Scrolling is a perceived barrier in the use of bring your own device (BYOD) to capture electronic patient reported outcomes (ePROs). This study explored the impact of scrolling on the measurement equivalence of electronic patient-reported outcome measures (ePROMs) in the presence and absence of scrolling. Methods Adult participants with a chronic condition involving daily pain completed ePROMs on four devices with different scrolling properties: a large provisioned device not requiring scrolling; two provisioned devices requiring scrolling – one with a “smart-scrolling” feature that disabled the “next” button until all information was viewed, and a second without this feature; and BYOD with smart-scrolling. The ePROMs included were the SF-12, EQ-5D-5L, and three pain measures: a visual analogue scale, a numeric response scale and a Likert scale. Participants completed English or Spanish versions according to their first language. Associations between ePROM scores were assessed using intraclass correlation coefficients (ICCs), with lower bound of 95% confidence interval (CI) > 0.7 indicating comparability. Results One hundred fifteen English- or Spanish-speaking participants (21-75y) completed all four administrations. High associations between scrolling and non-scrolling were observed (ICCs: 0.71–0.96). The equivalence threshold was met for all but one SF-12 domain score (bodily pain; lower 95% CI: 0.65) and two EQ-5D-5L item scores (pain/discomfort, usual activities; lower 95% CI: 0.64/0.67). Age, language, and device size produced insignificant differences in scores. Conclusions The measurement properties of PROMs are preserved even in the presence of scrolling on a handheld device. Further studies that assess scrolling impact over long-term, repeated use are recommended.

Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies

“…The benefits of BYOD studies are multifarious; the approach allows participants to use their own technologies, leading to better compliance [ 9 , 10 ] and, potentially, less chance of introducing biases, such as the Hawthorne effect, from monitoring technologies [ 11 ]. For participants, there is a familiarity with and access to the technology they use in their daily lives [ 12 , 13 ].…”

Regulatory Acceptance of Patient-Reported Outcome (PRO) Data from Bring-Your-Own-Device (BYOD) Solutions to Support Medical Product Labeling Claims

“…Remote PRO data capture can facilitate real-time understanding of a treatment’s safety and effectiveness by measuring symptoms and functional impacts in the participant’s usual environment instead of solely within a clinical setting [ 6 ]. Within the clinical trials industry, administering PRO instruments electronically (electronic patient-reported outcomes/ePRO) has become commonplace [ 7 – 9 ], especially when collecting data remotely at defined time points as it enables assurance of the contemporaneousness of the data. In addition, remote data capture can reduce both site and participant burden by enabling reduced frequency and duration of face-to-face clinic visits [ 3 ]; thus, offering the opportunity to facilitate increased adoption of the decentralized trial (DCT) model and enhance patient-centricity in clinical trials.…”

“…Historically, participants have been provisioned devices solely for the completion of ePROs. However, in recent years there has been increased interest in taking advantage of the option for participants to use their own device (smartphone, tablet, laptop) for completing study questionnaires—known as ‘bring-your-own-device’ (BYOD) [ 7 ].…”

“…It is important to note that when adopting a BYOD strategy in a clinical trial, it is recommended that sponsors also plan to have a percentage of provisioned devices available for use by study participants to ensure that participants who do not wish to use their own device or do not have a suitable device are still able to take part (and are not excluded from participating, creating bias in study findings) [ 7 ]. Furthermore, it is important to recognize that some participants may have concerns around data privacy and security associated with using their own device to submit health data [ 9 ], and thus may prefer to use a provisioned device.…”

“…Fortunately, we are now at a point as an industry where the evidence base, including multiple meta-analyses, has established that electronic versions of PRO instruments are comparable to paper versions (where comprised of standard item and response scales types, modifications made during migration are not substantial, and ePRO design best practices have been followed) [ 7 , 8 , 11 , 16 ]. Furthermore, studies specifically comparing data collected with BYOD to provisioned devices demonstrate equivalence, and this growing body of evidence supports the reliability and validity of PROs measured using BYOD solutions [ 7 , 8 , 11 , 17 , 18 ]. Thus, we hope that this can mitigate the concerns many sponsors still have around measurement equivalence of BYOD with paper and provisioned devices.…”