PRO Data Collection in Clinical Trials Using Mixed Modes: Report of the ISPOR PRO Mixed Modes Good Research Practices Task Force

Abstract

The objective of this report was to address the use and mixing of data collection modes within and between trials in which patient-reported outcome (PRO) end points are intended to be used to support medical product labeling. The report first addresses the factors that should be considered when selecting a mode or modes of PRO data collection in a clinical trial, which is often when mixing is first considered. Next, a summary of how to "faithfully" migrate instruments is presented followed by a section on qualitative and quantitative study designs used to evaluate measurement equivalence of the new and original modes of data collection. Finally, the report discusses a number of issues that must be taken into account when mixing modes is deemed necessary or unavoidable within or between trials, including considerations of the risk of mixing at different levels within a clinical trial program and mixing between different types of platforms. In the absence of documented evidence of measurement equivalence, it is strongly recommended that a quantitative equivalence study be conducted before mixing modes in a trial to ensure that sufficient equivalence can be demonstrated to have confidence in pooling PRO data collected by the different modes. However, we also strongly discourage the mixing of paper and electronic field-based instruments and suggest that mixing of electronic modes be considered for clinical trials and only after equivalence has been established. If proceeding with mixing modes, it is important to implement data collection carefully in the trial itself in a planned manner at the country level or higher and minimize ad hoc mixing by sites or individual subjects. Finally, when mixing occurs, it must be addressed in the statistical analysis plan for the trial and the ability to pool the data must be evaluated to then evaluate treatment effects with mixed modes data. A successful mixed modes trial requires a "faithful migration," measurement equivalence established between modes, and carefully planned implementation to minimize the risk of increased measurement error impacting the power of the trial to detect a treatment effect.

“Bring Your Own Device” (BYOD): The Future of Field-Based Patient-Reported Outcome Data Collection in Clinical Trials?

“…The ISPOR PRO Mixed Modes Good Research Practices Task Force recommended against mixing modes within a clinical trial unless there is documented evidence of measurement equivalence between or among the modes. 17 This recommendation is intended to avoid the introduction of measurement error that could reduce the ability to detect true treatment benefit. While this is primarily targeted at mixing paper and electronic data capture modes, the same concern could apply to mixing different modes of electronic data capture (eg, smartphone, tablet, personal computer, or even different brands of smartphone) if there are differences in the way the PRO instrument is rendered or presented.…”

“…However, the multiplicity of devices and OSs possible with BYOD-based data collection adds a layer of complexity and has the potential to introduce differences that reflect a moderate modification as defined by Eremenco et al 17 For example, it may be difficult to fit the same amount of text on the relatively small screen of a handheld device compared with the larger screen of a tablet or a desktop personal computer. This may force subjects using a smartphone to scroll down on a screen to view or read an item in its entirety, or items may end up being split over multiple screens.…”

Optimizing Electronic Capture of Clinical Outcome Assessment Data in Clinical Trials: The Case of Patient-Reported Endpoints

“…5 According to the report, mixing paper and electronic field-based assessments is the riskiest type of mode mixing because of the significant likelihood that the 2 modes will not generate equivalent responses. 5 As would be anticipated, deviations from the planned electronic collection of PRO data in a clinical trial primarily involve the study site or subject defaulting to a paper-based data collection form. This has the potential to introduce measurement error that could diminish the ability to document a true treatment benefit and should be avoided.…”

“…In addition, feasibility testing, which assesses whether the ePRO modality will work in the context of a specific study design (eg, multiple data entry sessions per day), should be considered on a case-by-case basis. 5 To ensure subjects are willing to use the ePRO device or system, the study informed consent form (ICF) should include a statement indicating use of an ePRO modality (eg, completion of a daily symptom diary using a handheld device) is a requirement for participation in the trial. If a subject is unwilling to use the ePRO modality after having read the ICF, then he or she does not meet inclusion criteria and should not be considered for participation in the study.…”

Streamlining the Validation of Patient Reported Outcome (PRO) Measures in Drug Regulatory Processes

“…Standards and best practices have recently been published on various aspects of collecting electronic PRO (ePRO) data, including on (1) the process of validating electronic systems used in the collection of ePRO for drug development processes and (2) the use and mixing of multiple modes of data collection, including ePRO, within and between clinical trials [30]. A challenging phenomena has come about as a result of the uptake of smart phones in the major pharmaceutical markets over the last 5-10 years, which has created new approaches for collecting ePRO, including the use of the subjects' own mobile devices.…”