Mixed methods grant applications in the health sciences: An analysis of reviewer comments

Abstract

Our aim was to understand how reviewers appraise mixed methods research by analyzing reviewer comments for grant applications submitted primarily to the National Institutes of Health. We requested scholars and consultants in the Mixed Methods Research Training Program (MMRTP) for the Health Sciences to send us summary statements from their mixed methods grant applications and obtained 40 summary statements of funded (40%) and unfunded (60%) mixed methods grant applications. We conducted a document analysis using a coding rubric based on the NIH Best Practices for Mixed Methods Research in the Health Sciences and allowed inductive codes to emerge. Reviewers favorably appraised mixed methods applications demonstrating coherence among aims and research design elements, detailed methods, plans for mixed methods integration, and the use of theoretical models. Reviewers identified weaknesses in mixed methods applications that lacked methodological details or rationales, had a high participant burden, and failed to delineate investigator roles. Successful mixed methods applications convey assumptions behind the methods chosen to accomplish specific aims and clearly detail the procedures to be taken. Investigators planning to use mixed methods should remember that reviewers are looking for both points of view.

Advancing discussion of ethics in mixed methods health services research

“…Ensuring MMR expertise on review boards, acting in a collaborative, feedback-based fashion [ 23 ], would facilitate guidance from IRBs on navigating ethical issues in MMR health services research. Given the increasing number of MMR projects in health services and implementation science [ 19 ], applications to ethics review boards provide an opportunity to gather empirical evidence on what strategies to promote ethical MMR are most feasible and effective. Most responsible conduct of research programs required for conducting health research (e.g., [ 24 ]) focus on quantitative or clinical research methods with limited or no mention of mixed methods designs.…”

“…Second, we analyzed NIH summary statements from grants submitted by MMRTP scholars and faculty and identified three inductive themes that emerged from the study section reviewer comments regarding ethical concerns. These included: (1) subject burden of participating in both qualitative and quantitative data collection; (2) participant confidentiality and anonymity, particularly risk of participant identification during the qualitative phase of the project or if video recordings were a data source; and (3) informed consent procedures that may require more explanation and thus be time-intensive [ 19 ]. Thus, our previous work with the MMRTP motivated our current study focus on understanding ethical considerations derived from practical experience in carrying out MMR.…”

Music in the acute preoperative nursing care — A mixed‐method pilot study

“…A mixed-method design inspired by Creswell was carried out using a convergent parallel design (Creswell, 2014;Guetterman et al, 2019).…”

Understanding no-show behaviour for cervical cancer screening appointments among hard-to-reach women in Bogotá, Colombia: A mixed-methods approach

“…According to Wisdom [21], the combined use of quantitative and qualitative methods can provide a more comprehensive picture of health services by capitalizing on the strengths of both approaches. Despite being a relatively new area, Guetterman et al [22] found that there is an increasing awareness of the relevance of mixed-methods research in order to address population and behavioural health problems. French et al [23], for example, used regression models to identify characteristics of children who missed their appointments and conducted phone interviews with GPs in order to understand their role and perceptions regarding low attendance levels.…”