Assessing User Engagement of an mHealth Intervention: Development and Implementation of the Growing Healthy App Engagement Index

Abstract

BackgroundChildhood obesity is an ongoing problem in developed countries that needs targeted prevention in the youngest age groups. Children in socioeconomically disadvantaged families are most at risk. Mobile health (mHealth) interventions offer a potential route to target these families because of its relatively low cost and high reach. The Growing healthy program was developed to provide evidence-based information on infant feeding from birth to 9 months via app or website. Understanding user engagement with these media is vital to developing successful interventions. Engagement is a complex, multifactorial concept that needs to move beyond simple metrics.ObjectiveThe aim of our study was to describe the development of an engagement index (EI) to monitor participant interaction with the Growing healthy app. The index included a number of subindices and cut-points to categorize engagement.MethodsThe Growing program was a feasibility study in which 300 mother-infant dyads were provided with an app which included 3 push notifications that was sent each week. Growing healthy participants completed surveys at 3 time points: baseline (T1) (infant age ≤3 months), infant aged 6 months (T2), and infant aged 9 months (T3). In addition, app usage data were captured from the app. The EI was adapted from the Web Analytics Demystified visitor EI. Our EI included 5 subindices: (1) click depth, (2) loyalty, (3) interaction, (4) recency, and (5) feedback. The overall EI summarized the subindices from date of registration through to 39 weeks (9 months) from the infant’s date of birth.Basic descriptive data analysis was performed on the metrics and components of the EI as well as the final EI score. Group comparisons used t tests, analysis of variance (ANOVA), Mann-Whitney, Kruskal-Wallis, and Spearman correlation tests as appropriate. Consideration of independent variables associated with the EI score were modeled using linear regression models.ResultsThe overall EI mean score was 30.0% (SD 11.5%) with a range of 1.8% - 57.6%. The cut-points used for high engagement were scores greater than 37.1% and for poor engagement were scores less than 21.1%. Significant explanatory variables of the EI score included: parity (P=.005), system type including “app only” users or “both” app and email users (P<.001), recruitment method (P=.02), and baby age at recruitment (P=.005).ConclusionsThe EI provided a comprehensive understanding of participant behavior with the app over the 9-month period of the Growing healthy program. The use of the EI in this study demonstrates that rich and useful data can be collected and used to inform assessments of the strengths and weaknesses of the app and in turn inform future interventions.

Controlled trial of an mHealth intervention to promote healthy behaviours in adolescence (TeenPower): Effectiveness analysis

“…Other authors reported retention rates in eHealth interventions ranging from 0.5% (Eysenbach, ) to 93% (Hammersley, Jones, & Okely, ). Therefore, user engagement in mHealth interventions is considered as the main concern of this type of intervention (Taki et al, ). The adolescents' lifestyle changes in both intervention and control groups differed according to the school district.…”

Key Lessons and Impact of the Growing Healthy mHealth Program on Milk Feeding, Timing of Introduction of Solids, and Infant Growth: Quasi-Experimental Study (Preprint)

“…Our quantitative analysis of app usage [41] revealed higher engagement with the program amongst those recruited by their health practitioner, those who registered when their infant was younger, those who were first time mothers, and those using both the app and website (via email links) compared to those using the app alone. This suggests that to maximise engagement and potential impact, consideration should be given in the future to focus on recruiting first time parents, with the help of health practitioners during the early postnatal or antenatal period.…”

A Co-Designed, Culturally-Tailored mHealth Tool to Support Healthy Lifestyles in Māori and Pasifika Communities in New Zealand: Protocol for a Cluster Randomized Controlled Trial

“…At 12-week follow-up, user engagement and interaction with the mobile phone app will be quantified using an engagement index [44]. The index is an adapted version of the Web Analytics Demystified Visitor Engagement Index [45].…”