What do healthcare professionals need to turn risk models for type 2 diabetes into usable computerized clinical decision support systems? Lessons learned from the MOSAIC project

Abstract

Background To understand user needs, system requirements and organizational conditions towards successful design and adoption of Clinical Decision Support Systems for Type 2 Diabetes (T2D) care built on top of computerized risk models. Methods The holistic and evidence-based CEHRES Roadmap , used to create eHealth solutions through participatory development approach, persuasive design techniques and business modelling, was adopted in the MOSAIC project to define the sequence of multidisciplinary methods organized in three phases, user needs , implementation and evaluation . The research was qualitative, the total number of participants was ninety, about five-seventeen involved in each round of experiment. Results Prediction models for the onset of T2D are built on clinical studies, while for T2D care are derived from healthcare registries. Accordingly, two set of DSSs were defined: the first, T2D Screening , introduces a novel routine; in the second case, T2D Care , DSSs can support managers at population level, and daily practitioners at individual level. In the user needs phase , T2D Screening and solution T2D Care at population level share similar priorities, as both deal with risk-stratification. End-users of T2D Screening and solution T2D Care at individual level prioritize easiness of use and satisfaction, while managers prefer the tools to be available every time and everywhere. In the implementation phase , three Use Cases were defined for T2D Screening , adapting the tool to different settings and granularity of information. Two Use Cases were defined around solutions T2D Care at population and T2D Care at individual , to be used in primary or secondary care. Suitable filtering options were equipped with “attractive” visual analytics to focus the attention of end-users on specific parameters and events. In the evaluation phase , good levels of user experience versus bad level of usability suggest that end-users of T2D Screening perceived the potential, but they are worried about complexity. Usability and user experience were above acceptable thresholds for T2D Care at population and T2D Care at individual . Conclusions By using a holistic approach, we have been able to understand user needs, behaviours and interactions and give new insights in the definition of effective Decision Support Systems to deal with the complexity of T2D care. Electronic supplementary material T...

Methods for Human-Centered eHealth Development: Narrative Scoping Review

“…Focus groups can be used in a sequential way: multiple focus groups can be conducted in a row and the goal and content of each focus group can be based on the outcomes of the previous focus groups. However, this iterative approach was said to be quite time consuming for researchers and participants [ 33 , 54 , 64 , 82 , 84 - 86 , 90 , 92 ]. Third, to ensure that valuable information for eHealth development is gathered, the content and form of focus groups need to be adapted, based on the topic and target group [ 39 - 41 , 51 , 55 , 61 , 87 ].…”

“…First, multiple authors indicated that usability tests should be conducted with a broad range of stakeholders: not only end users such as patients, but also caregivers, managers, technology developers, and experts on content and design [ 21 , 43 , 54 , 64 , 77 , 112 , 127 , 131 , 152 , 155 ]. Different types of participants can provide different kinds of feedback on an eHealth technology.…”

“…It was suggested that experts on, for example, usability or persuasive design can be included by means of cognitive walkthroughs or heuristic evaluations, whereas users can be involved through think-aloud procedures, guided by scenarios [ 127 ]. Second, authors stated that ideally, multiple methods should be combined to paint a full picture of a prototype’s usability [ 24 , 25 , 64 , 77 , 80 , 118 , 152 ]; for example, qualitative methods such as think-aloud procedures and interviews can be combined. Qualitative approaches can also be combined with quantitative data collected by means of, for example, log data analyses, questionnaire data, or eye tracking [ 127 ].…”

“…First, authors stated that values should be formulated in such a way that they are neither too specific to prevent overlap with requirements nor too broad and vague. A shared understanding among the eHealth development team members about what values are is essential to achieve this [ 15 , 64 , 97 , 152 ]. Second, values should capture the whole range of stakeholder needs and wishes, not merely those of end users of the eHealth technology [ 81 , 129 ].…”

ML-MEDIC: A Preliminary Study of an Interactive Visual Analysis Tool Facilitating Clinical Applications of Machine Learning for Precision Medicine

“…The current evaluation of ML-MEDIC is limited to more of a qualitative assessment of the benefit of GUI-based, user-interactive tools by the clinical community to facilitate and improve the use of ML in clinical applications. Further testing and quantitative analysis is needed to assess resources such as ML-MEDIC and the potential for use in clinical decision support and various clinical research applications [ 31 ].…”