Usability Testing of a Social Media Chatbot for Increasing Physical Activity Behavior

Abstract

Digital interventions for increasing physical activity behavior have shown great potential, especially those with social media. Chatbots, also known as conversational agents, have emerged in healthcare in relation to digital interventions and have proven effective in promoting physical activity among adults. The study’s objective is to explore users’ experiences with a social media chatbot. The concept and the prototype development of the social media chatbot MYA were realized in three steps: requirement analysis, concept development, and implementation. MYA’s design includes behavior change techniques effective in increasing physical activity through digital interventions. Participants in a usability study answered a survey with the Chatbot Usability Questionnaire (CUQ), which is comparable to the Systems Usability Scale. The mean CUQ score was below 68, the benchmark for average usability. The highest mean CUQ score was 64.5 for participants who thought MYA could help increase their physical activity behavior. The lowest mean CUQ score was 40.6 for participants aged between 50 and 69 years. Generally, MYA was considered to be welcoming, very easy to use, realistic, engaging, and informative. However, some technical issues were identified. A good and diversified user experience promotes prolonged chatbot use. Addressing identified issues will enhance users’ interaction with MYA.

Developing a Technical-Oriented Taxonomy to Define Archetypes of Conversational Agents in Health Care: Literature Review and Cluster Analysis

“…The setting perspective comprises 3 dimensions and characterizes how a CA is embedded in the environment of a user. The context illustrates whether the CA serves a specific domain or can interact on any topic with its users [ 2 , 3 , 28 ]. Service duration [ 4 , 29 ] refers to the services offered by a CA.…”

“…In the health domain, CAs became popular as they allow for simulating the real-life experience in a health care setting, which is the conversation with a physician. They have been tested for patient education [ 2 ], supporting behavior change [ 3 ], or delivering cognitive behavioral therapy [ 4 ]. Furthermore, hospitals and private clinics use CAs to triage and clerk patients even before they come into the consulting room.…”

Examining Patient Engagement in Chatbot Development Approaches for Healthy Lifestyle and Mental Wellness Interventions: Scoping Review

“…Three of the included studies were RCTs [ 31 , 35 , 42 ], 4 were nonrandomized experimental studies [ 32 , 34 , 36 , 41 ], 3 were user-design and development studies [ 30 , 37 , 40 ], 3 were usability studies [ 38 , 39 , 43 ], 1 was a feasibility study [ 28 ], and 2 were proof-of-concept studies [ 29 , 33 ].…”

“…Where reported, the inclusion of female participants ranged from 31.4% to 100% [ 37 ]. Five studies involved an exercise intervention [ 34 , 35 , 37 , 39 , 43 ]. Three studies included a mental wellness intervention for healthy coping, life skill coaching, and positive psychology skill building [ 29 - 31 ].…”

Development, Implementation, and Usability Evaluation of a Chatbot to Effectively Provide Information on the Human Papillomavirus Vaccine

“…The instrument has been extensively tested and compared with the SUS and UEQ measures, with a fair degree of aggreement with prior scales. Several studies have employed the CUQ measure for assessment of chatbot usability [35], [36]. On account of its specialization to chatbots, coupled with its extensive use, we elected to use the CUQ measure to evaluate the usability of GARDI.…”