How do Medical Societies Select Science for Conference Presentation? How Should They?

Abstract

IntroductionNothing has been published to describe the practices of medical societies in choosing abstracts for presentations at their annual meetings. We surveyed medical societies to determine their practices, and also present a theoretical analysis of the topic.MethodsWe contacted a convenience sample of large U.S. medical conferences, and determined their approach to choosing abstracts. We obtained information from web sites, telephone, and email. Our theoretical analysis compares values-based and empirical approaches for scoring system development.ResultsWe contacted 32 societies and obtained data on 28 (response rate 88%). We excluded one upon learning that research was not presented at its annual meeting, leaving 27 for analysis. Only 2 (7%) made their abstract scoring process available to submitters. Reviews were blinded in most societies (21;78%), and all but one asked reviewers to recuse themselves for conflict of interest (96%). All required ≥3 reviewers. Of the 24 providing information on how scores were generated, 21 (88%) reported using a single gestalt score, and three used a combined score created from pooled domain-specific sub-scores. We present a framework for societies to use in choosing abstracts, and demonstrate its application in the development of a new scoring system.ConclusionsMost medical societies use subjective, gestalt methods to select research for presentation at their annual meetings and do not disclose to submitters the details of how abstracts are chosen. We present a new scoring system that is transparent to submitters and reviewers alike with an accompanying statement of values and ground rules. We discuss the challenges faced in selecting abstracts for a large scientific meeting and share the values and practical considerations that undergird the new system.

Publication of oral and video presentations from the Society of Gynecologic Oncology annual meeting over 11 years – What characteristics were important?

“…The authors explained that international authors may be more likely to misunderstand the medical specialty addressed at a US conference, and therefore inappropriately submit their work. This would lead to more rejections of international abstracts ( Kuczmarski et al, 2015 ). On the other hand, a prior report regarding cardiovascular disease research showed that international collaboration has been increasing, especially in the European Union.…”

Conversion of Skeletal Society of Radiology annual meeting abstracts to publications in 2010–2015

“…This study found that based on the mean scores given for each presentation, 17–20% of subjective decisions for acceptance or rejection would have been reversed ( Blackburn & Hakel, 2006 ). Kuczmarski et al surveyed 27 large scientific conferences to evaluate the screening process for conference abstracts to be presented and found that only two (7%) made the scoring process available to submitters and the public ( Kuczmarski, Raja & Pallin, 2015 ). As a follow-up to the survey, they created a comprehensive scoring system that emphasized transparency and objectivity.…”

“…This scoring system asks reviewers to provide 0 to 2 points for each of the seven categories that are outlined by general criteria for each grade, as well as specific examples. We suggest that the SSR takes steps to ensure transparency in their scoring process that may result in improved abstract quality ( Kuczmarski, Raja & Pallin, 2015 )…”

Leveling the field: Development of reliable scoring rubrics for quantitative and qualitative medical education research abstracts

“…2,4,5 Medical conferences often apply rubrics designed for judging clinical or basic science submissions, which reflect standard hypothesis-testing methods and often use a single subjective Gestalt rating for quality decisions. 6 This may result in the systematic exclusion of studies that employ alternate, but equally rigorous methods, such as research in medical education. Existing scoring systems, commonly designed for biomedical research, may not accurately assess the scope, methods, and types of results commonly reported in medical education research abstracts, which may lead to a disproportionately high rate of rejection of these abstracts.…”