Physicians’ perception of alternative displays of clinical research evidence for clinical decision support – A study with case vignettes

Abstract

Objective To design alternate information displays that present summaries of clinical trial results to clinicians to support decision-making; and to compare the displays according to efficacy and acceptability. Methods A 6-between (information display presentation order) by 3-within (display type) factorial design. Two alternate displays were designed based on Information Foraging theory: a narrative summary that reduces the content to a few sentences; and a table format that structures the display according to the PICO (Population, Intervention, Comparison, Outcome) framework. The designs were compared with the summary display format available in PubMed. Physicians were asked to review five clinical studies retrieved for a case vignette; and were presented with the three display formats. Participants were asked to rate their experience with each of the information displays according to a Likert scale questionnaire. Results Twenty physicians completed the study. Overall, participants rated the table display more highly than either the text summary or PubMed’s summary format (5.9 vs. 5.4 vs. 3.9 on a scale between 1 [strongly disagree] and 7 [strongly agree]). Usefulness ratings of seven pieces of information, i.e. patient population, patient age range, sample size, study arm, primary outcome, results of primary outcome, and conclusion, were high (average across all items = 4.71 on a 1 to 5 scale, with 1=not at all useful and 5=very useful). Study arm, primary outcome, and conclusion scored the highest (4.9, 4.85, and 4.85 respectively). Participants suggested additional details such as rate of adverse effects. Conclusion The table format reduced physicians’ perceived cognitive effort when quickly reviewing clinical trial information and was more favorably received by physicians than the narrative summary or PubMed’s summary format display.

Interactive Visual Displays for Interpreting the Results of Clinical Trials: Formative Evaluation With Case Vignettes (Preprint)

“…Our study findings add to the growing evidence supporting alternative information display formats to convey the gist of clinical studies [11][12][13][14][15][16][17][18][19][20][21][22], suggesting that the standard format of scientific reporting, especially for article abstracts, is worth reconsidering. The ideal abstract display format should match clinicians' mental model to reduce cognitive workload in interpreting clinical study results.…”

“…All participants received a $100 incentive to participate in the study sessions. A previous study with similar design [22] demonstrated that 20 participants are enough to detect a moderate difference between interactive visual displays and narrative display with a power of 0.80.…”

“…The questions (available in the online supplement) were adapted from the System Usability Scale (SUS) [42], the National Aeronautics and Space Administration (NASA) Task Load Index (NASA-TLX) tool [43], and from Slager et al [22] Two versions of the questionnaire were used, depending on which tool the participant was randomly assigned to use first. Each participant only needed to complete one survey.…”

“…Building on Slager et al 's exploratory study on static tabular displays [22], we employed information foraging theory [23] and information visualization techniques to design a high-fidelity prototype, with interactive visual displays of RCT results. The information displays were designed to help clinicians rapidly review, synthesize, and compare the results of relevant RCTs for the treatment of a specific patient.…”

Interactive Visual Displays for Interpreting the Results of Clinical Trials: Formative Evaluation With Case Vignettes

“…Overall, we hypothesized that interactive visual displays would reduce clinicians’ cognitive workload in interpreting RCTs compared with narrative RCT abstracts. Building on Slager et al’s exploratory study on static tabular displays [ 22 ], we employed information foraging theory [ 23 ] and information visualization techniques to design a high-fidelity prototype with interactive visual displays of RCT results. The information displays were designed to help clinicians rapidly review, synthesize, and compare the results of relevant RCTs for the treatment of a specific patient.…”