Development and Evaluation of an Evaluation Tool for Healthcare Smartphone Applications

Jin, Meiling; Kim, Jeongeun

doi:10.1089/tmj.2014.0151

Cited by 47 publications

(39 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They suggested a customizable Evernote-based system for reproduction by interested students [ 20 ]. Recently, Jin and Kim [ 21 ] described their three-phases evaluation of a “Tool for Healthcare Smartphone Applications”. They concluded that the evaluation tool they developed and tested in their study was an appropriate and widely applicable tool to evaluate health care mobile phone apps to determine if they are reliable and useful [ 21 ].…”

Section: Discussionmentioning

confidence: 99%

Mobile Phone Use Among Medical Residents: A Cross-Sectional Multicenter Survey in Saudi Arabia

Jamal

Temsah²,

Khan³

et al. 2016

JMIR Mhealth Uhealth

View full text Add to dashboard Cite

BackgroundMobile phones have great potential for medical education, as they allow health care providers and students to access resources efficiently at the precise time at the point-of-care to help in informed decision making.ObjectiveThe objective of the study was to evaluate the prevalence of mobile phone usage among medical residents and to explore their attitudes, perceptions, and the challenges they experience when using mobile phones in academic and clinical practice.MethodsA cross-sectional survey was conducted on all 133 residents in 17 different specialties across two large academic hospitals in Riyadh, Saudi Arabia. The Web-based validated questionnaire measured mobile phone platform preferences, and their uses in general and medical practice. The perception of confidentiality and safety impact of using mobile phones for communication and accessing patient’s data was also explored, alongside challenges of use and how residents learn to use their mobile phone.ResultsWith a response rate of 101/133 (75.9%) and mean age of 27.8 (SD 3.0) years, we found that 100/101 (99.0%) of participants were mobile phone users with mean duration of use of 5.12 (SD 2.4) years, and a range from 1 to 12 years. There was no significant difference in use between male and female respondents. A negative linear correlation was found between age and use duration (P=.004). The most common operating system used by participants was the iOS platform (55/101, 54.5%), with English the most commonly used language to operate residents’ mobile phones (96/100, 96.0%) despite their native language being Arabic. For communication outside medical practice, chatting applications such as WhatsApp matched phone calls as most commonly used tools (each 88/101, 87.1%). These were also the primary tools for medical communication, but used at a lower rate (each 65/101, 64.4%). In medical practice, drug (83/101, 82.2%) and medical (80/101, 79.2%) references and medical calculation applications (61/101, 60.4%) were the most commonly used. Short battery life (48/92, 52%) was the most common technical difficulty, and distraction at least on a weekly basis (54/92, 58%) was the most likely side effect of using a mobile phone in medical practice. Practically, all participants agreed with the idea of integrating medical staff mobile phones with the hospital information system. Most residents described themselves as self-learners, while half learned from peers, and a quarter learned from the Internet. Only 7/101 (6.9%) had received formal training on the medical use of mobile phones. Over half of residents thought it was safe to discuss patients over their personal, nonencrypted email.ConclusionsMobile phone use among medical residents has become almost universal in academic and clinical settings. Thus, academic and health care institutions should support proper utilization of these devices in medical training and point-of-care decision making, while continuing to protect patient confidentiality.

show abstract

Section: Discussionmentioning

confidence: 99%

Mobile Phone Use Among Medical Residents: A Cross-Sectional Multicenter Survey in Saudi Arabia

Jamal

Temsah²,

Khan³

et al. 2016

JMIR Mhealth Uhealth

View full text Add to dashboard Cite

show abstract

“…82 Such omissions pose risks to patient safety and explicit app risk assessment has been proposed. 15 Various rating scales, 83,84 scoring systems, 85,86 checklists, [87][88][89][90][91] toolkits, 92,93 questionnaires, 94 development guides 95 and reporting standards for app studies 96,97 all promote more transparent and objective reporting and evaluation. The existence of so many instruments suggests no single one meets all needs.…”

Section: Objective No 3: Methods For Evaluating App Quality and Utilitymentioning

confidence: 99%

Going digital: a narrative overview of the effects, quality and utility of mobile apps in chronic disease self-management

Scott

Scuffham²,

Gupta

et al. 2020

Aust. Health Review

View full text Add to dashboard Cite

Objective Smartphone health applications (apps) are being increasingly used to assist patients in chronic disease self-management. The effects of such apps on patient outcomes are uncertain, as are design features that maximise usability and efficacy, and the best methods for evaluating app quality and utility. Methods In assessing efficacy, PubMed, Cochrane Library and EMBASE were searched for systematic reviews (and single studies if no systematic review was available) published between January 2007 and January 2018 using search terms (and synonyms) of ‘smartphone’ and ‘mobile applications’, and terms for each of 11 chronic diseases: asthma, chronic obstructive lung disease (COPD), diabetes, chronic pain, serious mental health disorders, alcohol and substance addiction, heart failure, ischaemic heart disease, cancer, cognitive impairment, chronic kidney disease (CKD). With regard to design features and evaluation methods, additional reviews were sought using search terms ‘design’, ‘quality,’ ‘usability’, ‘functionality,’ ‘adherence’, ‘evaluation’ and related synonyms. Results Of 13 reviews and six single studies assessing efficacy, consistent evidence of benefit was seen only with apps for diabetes, as measured by decreased glycosylated haemoglobin levels (HbA1c). Some, but not all, studies showed benefit in asthma, low back pain, alcohol addiction, heart failure, ischaemic heart disease and cancer. There was no evidence of benefit in COPD, cognitive impairment or CKD. In all studies, benefits were clinically marginal and none related to morbid events or hospitalisation. Twelve design features were identified as enhancing usability. An evaluation framework comprising 32 items was formulated. Conclusion Evidence of clinical benefit of most available apps is very limited. Design features that enhance usability and maximise efficacy were identified. A provisional ‘first-pass’ evaluation framework is proposed that can help decide which apps should be endorsed by government agencies following more detailed technical assessments and which could then be recommended with confidence by clinicians to their patients. What is known about the topic? Smartphone health apps have attracted considerable interest from patients and health managers as a means of promoting more effective self-management of chronic diseases, which leads to better health outcomes. However, most commercially available apps have never been evaluated for benefits or harms in clinical trials, and there are currently no agreed quality criteria, standards or regulations to ensure health apps are user-friendly, accurate in content, evidence based or efficacious. What does this paper add? This paper presents a comprehensive review of evidence relating to the efficacy, usability and evaluation of apps for 11 common diseases aimed at assisting patients in self-management. Consistent evidence of benefit was only seen for diabetes apps; there was absent or conflicting evidence of benefit for apps for the remaining 10 diseases. Benefits that were detected were of marginal clinical importance, with no reporting of hard clinical end-points, such as mortality or hospitalisations. Only a minority of studies explicitly reported using behaviour change theories to underpin the app intervention. Many apps lacked design features that the literature identified as enhancing usability and potential to confer benefit. Despite a plethora of published evaluation tools, there is no universal framework that covers all relevant clinical and technical attributes. An inclusive list of evaluation criteria is proposed that may overcome this shortcoming. What are the implications for practitioners? The number of smartphone apps will continue to grow, as will the appetite for patients and clinicians to use them in chronic disease self-management. However, the evidence to date of clinical benefit of most apps already available is very limited. Design features that enhance usability and clinical efficacy need to be considered. In making decisions about which apps should be endorsed by government agencies and recommended with confidence by clinicians to their patients, a comprehensive but workable evaluation framework needs to be used by bodies assuming the roles of setting and applying standards.

show abstract

“…The questions were based on a tool developed by Jin and Kim (2015) to evaluate information and instruction clarity, interface intuitiveness and data security of health-care apps. 10 One survey was sent per family via parents' emails or by post if parents did not have a valid email address.…”

Section: Parents' Perceptionsmentioning

confidence: 99%

The Baby Moves smartphone app for General Movements Assessment: Engagement amongst extremely preterm and term‐born infants in a state‐wide geographical study

Kwong

Eeles

Olsen

et al. 2018

J Paediatrics Child Health

View full text Add to dashboard Cite

Aim The Baby Moves smartphone application is designed for parents to video their infants' spontaneous movement for remote General Movements Assessment (GMA). We aimed to assess the engagement with Baby Moves amongst high‐ and low‐risk infants' families and the socio‐demographic variables related to engagement. Methods Families of extremely preterm (EP; <28 weeks' gestational age) or extremely low‐birthweight (ELBW; <1000 g) infants and term‐born controls from a state‐wide geographical cohort study were asked to download Baby Moves. Baby Moves provided reminders and instructions to capture videos of their infants' general movements. Parents were surveyed about Baby Moves' usability. Results The parents of 451 infants (226 EP/ELBW; 225 control) were recruited; 416 (204 EP/ELBW; 212 control) downloaded Baby Moves, and 346 (158 EP/ELBW; 188 control) returned at least one scorable video for remote GMA. Fewer EP/ELBW families submitted a scorable video than controls (70 vs. 83%, respectively; odds ratio (OR) 0.48, 95% confidence interval (CI) 0.3–0.79, P = 0.003), but the difference diminished when adjusted for socio‐demographic variables (OR 1.09, 95% CI 0.59–2.0, P = 0.79). Families who received government financial support (OR 0.28, 95% CI 0.1–0.78, P = 0.015), who spoke limited English at home (OR 0.39, 95% CI 0.22–0.69, P = 0.001) or with lower maternal education (OR 0.38, 95% CI 0.21–0.68, P = 0.001) were less likely to return a scorable video. Surveyed parents responded mostly positively to Baby Moves' usability. Conclusions Most parents in this study successfully used Baby Moves to capture infant movements for remote GMA. Families of lower socio‐demographic status used Baby Moves less.

show abstract

Development and Evaluation of an Evaluation Tool for Healthcare Smartphone Applications

Cited by 47 publications

References 6 publications

Mobile Phone Use Among Medical Residents: A Cross-Sectional Multicenter Survey in Saudi Arabia

Mobile Phone Use Among Medical Residents: A Cross-Sectional Multicenter Survey in Saudi Arabia

Going digital: a narrative overview of the effects, quality and utility of mobile apps in chronic disease self-management

The Baby Moves smartphone app for General Movements Assessment: Engagement amongst extremely preterm and term‐born infants in a state‐wide geographical study

Contact Info

Product

Resources

About