No abstract

Crowdsourcing shifts medical research from a closed environment to an open collaboration between the public and researchers. We define crowdsourcing as an approach to problem solving which involves an organization having a large group attempt to solve a problem or part of a problem, then sharing solutions. Crowdsourcing allows large groups of individuals to participate in medical research through innovation challenges, hackathons, and related activities. The purpose of this literature review is to examine the definition, concepts, and applications of crowdsourcing in medicine. This multi-disciplinary review defines crowdsourcing for medicine, identifies conceptual antecedents (collective intelligence and open source models), and explores implications of the approach. Several critiques of crowdsourcing are also examined. Although several crowdsourcing definitions exist, there are two essential elements: (1) having a large group of individuals, including those with skills and those without skills, propose potential solutions; (2) sharing solutions through implementation or open access materials. The public can be a central force in contributing to formative, pre-clinical, and clinical research. A growing evidence base suggests that crowdsourcing in medicine can result in high-quality outcomes, broad community engagement, and more open science.

Both sex (biological factors) and gender (socio-cultural factors) shape health. To produce the best possible health research evidence, it is essential to integrate sex and gender considerations throughout the research process. Despite growing recognition of the importance of these factors, progress towards sex and gender integration as standard practice has been both slow and uneven in health research. In this commentary, we examine the challenges of integrating sex and gender from the research perspective, as well as strategies that can be used by researchers, funders and journal editors to address these challenges. Barriers to the integration of sex and gender in health research include problems with inconsistent terminology, difficulties in applying the concepts of sex and gender, failure to recognise the impact of sex and gender, and challenges with data collection and datasets. We analyse these barriers as strategic points of intervention for improving the integration of sex and gender at all stages of the research process. To assess the relative success of these strategies in any given study, researchers, funders and journal editors would benefit from a tool to evaluate the quality of sex and gender integration in order to establish benchmarks in research excellence. These assessment tools are needed now amidst growing institutional recognition that both sex and gender are necessary elements for advancing the quality and utility of health research evidence.

Background: Crowdsourcing is used increasingly in health and medical research. Crowdsourcing is the process of aggregating crowd wisdom to solve a problem. The purpose of this systematic review is to summarize quantitative evidence on crowdsourcing to improve health. Methods: We followed Cochrane systematic review guidance and systematically searched seven databases up to September 4th 2019. Studies were included if they reported on crowdsourcing and related to health or medicine. Studies were excluded if recruitment was the only use of crowdsourcing. We determined the level of evidence associated with review findings using the GRADE approach. Results: We screened 3508 citations, accessed 362 articles, and included 188 studies. Ninety-six studies examined effectiveness, 127 examined feasibility, and 37 examined cost. The most common purposes were to evaluate surgical skills (17 studies), to create sexual health messages (seven studies), and to provide layperson cardiopulmonary resuscitation (CPR) out-of-hospital (six studies). Seventeen observational studies used crowdsourcing to evaluate surgical skills, finding that crowdsourcing evaluation was as effective as expert evaluation (low quality). Four studies used a challenge contest to solicit human immunodeficiency virus (HIV) testing promotion materials and increase HIV testing rates (moderate quality), and two of the four studies found this approach saved money. Three studies suggested that an interactive technology system increased rates of layperson initiated CPR out-ofhospital (moderate quality). However, studies analyzing crowdsourcing to evaluate surgical skills and laypersoninitiated CPR were only from high-income countries. Five studies examined crowdsourcing to inform artificial intelligence projects, most often related to annotation of medical data. Crowdsourcing was evaluated using different outcomes, limiting the extent to which studies could be pooled. Conclusions: Crowdsourcing has been used to improve health in many settings. Although crowdsourcing is effective at improving behavioral outcomes, more research is needed to understand effects on clinical outcomes and costs. More research is needed on crowdsourcing as a tool to develop artificial intelligence systems in medicine.

IntroductionStakeholder engagement is an essential component of HIV clinical trials. We define stakeholder engagement as an input by individuals or groups with an interest in HIV clinical trials to inform the design or conduct of said trials. Despite its value, stakeholder engagement to inform HIV clinical trials has not been rigorously examined. The purpose of our systematic review is to examine stakeholder engagement for HIV clinical trials and compare it to the recommendations of the UNAIDS/AVAC Good Participatory Practice (GPP) guidelines.MethodsWe used the PRISMA checklist and identified English language studies describing stakeholder engagement to inform HIV clinical trials. Four databases (PubMed, Ovid, CINAHL and Web of Science) and six journals were searched, with additional studies identified using handsearching and expert input. Two independent reviewers examined citations, abstracts and full texts. Data were extracted on country, engagement methods, stakeholder types and purpose of stakeholder engagement. Based on the GPP guidelines, we examined how frequently stakeholder engagement was conducted to inform clinical trial research question development, protocol development, recruitment, enrolment, follow‐up, results and dissemination.Results and discussionOf the 917 citations identified, 108 studies were included in the analysis. Forty‐eight studies (44.4%) described stakeholder engagement in high‐income countries, thirty (27.8%) in middle‐income countries and nine (8.3%) in low‐income countries. Fourteen methods for stakeholder engagement were identified, including individual (e.g. interviews) and group (e.g. community advisory boards) strategies. Thirty‐five types of stakeholders were engaged, with approximately half of the studies (60; 55.6%) engaging HIV‐affected community stakeholders (e.g. people living with HIV, at‐risk or related populations of interest). We observed greater frequency of stakeholder engagement to inform protocol development (49 studies; 45.4%) and trial recruitment (47 studies; 43.5%). Fewer studies described stakeholder engagement to inform post‐trial processes related to trial results (3; 2.8%) and dissemination (11; 10.2%).ConclusionsOur findings identify important directions for future stakeholder engagement research and suggestions for policy. Most notably, we found that stakeholder engagement was more frequently conducted to inform early stages of HIV clinical trials compared to later stages. In order to meet recommendations established in the GPP guidelines, greater stakeholder engagement across all clinical trial stages is needed.