BACKGROUND There is a growing trend in the potential benefits and application of log data for the evaluation of mHealth Apps. However, the process by which insights may be derived from log data remains unstructured, resulting in underutilisation of mHealth data. OBJECTIVE We aimed to acquire an understanding of how log data analysis can be used to generate valuable insights in support of realistic evaluations of mobile Apps through a scoping review. This understanding is delineated according to publication trends, associated concepts and characteristics of log data, framework or processes required to develop insights from log data, and how these insights may be utilised towards evaluation of Apps. METHODS The PRISMA-ScR guidelines for a scoping review were followed. The Scopus database, the Journal of Medical Internet Research (JMIR), and grey literature (through a Google search) delivered 105 articles of which 33 articles were retained in the sample for analysis and synthesis. RESULTS A definition for log data is developed from its characteristics and articulated as: anonymous records of users’ real-time interactions with the application, collected objectively or automatically and often accessed from cloud-based storage. Publications for theoretical and empirical work on log data analysis have increased between 2010 and 2021 (100% and 95% respectively). The research approach is distributed between inductive (43%), deductive (30%), and a hybrid approach (27%). Research methods include mixed-methods (73%) and quantitative only (27%), although mixed-methods dominate since 2018. Only 30% of studies articulated the use of a framework or model to perform the log data analysis. Four main focus areas for log data analysis are identified as usability (40%), engagement (15%), effectiveness (15%), and adherence (15%). An average of one year of log data is used for analysis, with an average of three years from the launch of the App to the analysis. Collected indicators include user events or clicks made, specific features of the App, and timestamps of clicks. The main calculated indicators are features used or not used (24/33), frequency (21/33), and duration (18/33). Reporting the calculated indicators per ‘user or user group’ was the most used reference point. CONCLUSIONS Standardised terminology, processes, frameworks, and explicit benchmarks to utilise log data are lacking in literature. Thereby, the need for a conceptual framework that is able to standardise the log analysis of mobile Apps is determined. We provide a summary of concepts towards such a framework. CLINICALTRIAL NA

There is a growing trend in the potential benefits and application of log data to evaluate mHealth Apps. However, the process by which log data is used to derive insights remains unstructured, resulting in the underutilization of mHealth data. We aimed to explore extant literature and guidance through a scoping review on how log data analysis can be used to generate valuable insights in support of the evaluation of mobile Apps. The scoping review followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines for a scoping review. The Scopus database and grey literature (through a Google search) delivered 105 articles, and we applied inclusion and exclusion criteria to retain 33 articles in the sample for analysis and synthesis. This scoping review sought to identify how log data are used for mobile App evaluations. By highlighting the existing trends found in literature, identifying the similarities and differences between mHealth and General App analyses, and categorizing the indicators, insights, and improvements, this study contributes to the existing knowledge base of mHealth evaluations and future standardizations. The concepts and categories identified by this review are combined to form a proposed conceptual framework that will be refined and incorporated into future research toward addressing the gap identified in the current literature.

No abstract