Exploring How Professionals Within Agile Health Care Informatics Perceive Visualizations of Log File Analyses: Observational Study Followed by a Focus Group Interview

Abstract

Background An increasing number of software companies work according to the agile software development method, which is difficult to integrate with user-centered design (UCD) practices. Log file analysis may provide opportunities for integrating UCD practices in the agile process. However, research within health care information technology mostly has a theoretical approach and is often focused on the researcher’s interpretation of log file analyses. Objective We aimed to propose a systematic approach to log file analysis in this study and present this to developers to explore how they react and interpret this approach in the context of a real-world health care information system, in an attempt to answer the following question: How may log file analyses contribute to increasing the match between the health care system and its users, within the agile development method, according to agile team members? Methods This study comprised 2 phases to answer the research question. In the first phase, log files were collected from a health care information system and subsequently analyzed (summarizing sequential patterns, heat mapping, and clustering). In the second phase, the results of these analyses are presented to agile professionals during a focus group interview. The interpretations of the agile professionals are analyzed by open axial coding. Results Log file data of 17,924 user sessions and, in total, 176,678 activities were collected. We found that the Patient Timeline is mainly visited, with 23,707 (23,707/176,678; 13.42%) visits in total. The main unique user session occurred in 5.99% (1074/17,924) of all user sessions, and this comprised Insert Measurement Values for Patient and Patient Timeline, followed by the page Patient Settings and, finally, Patient Treatment Plan. In the heat map, we found that users often navigated to the pages Insert Measurement Values and Load Messages Collaborate. Finally, in the cluster analysis, we found 5 clusters, namely, the Information-seeking cluster, the Collaborative cluster, the Mixed cluster, the Administrative cluster, and the Patient-oriented cluster. We found that the interpretations of these results by agile professionals are related to stating hypotheses (n=34), comparing paths (n=31), benchmarking (n=22), and prioritizing (n=17). Conclusions We found that analyzing log files provides agile professionals valuable insights into users’ behavior. Therefore, we argue that log file analyses should be used within agile development to inform professionals about users’ behavior. In this way, further UCD research can be informed by these results, making the methods less labor intensive. Moreover, we argue that these translations to an approach for further UCD research will be carried out by UCD specialists, as they are able to infer which goals the user had when going through these paths when looking at the log data.

Outpatient portal use among pregnant individuals: Cross-sectional, temporal, and cluster analysis of use

“…However, the scale at which this activity information exists has been harnessed to summarize sequences of activity and further identify groups of common sequential actions. [20][21][22][23] Processing audit log file data presents a challenge; however, recent studies have developed, standardized, and shared such methods for processing patient portal audit log files and measuring patient portal use. [24][25][26] These studies were performed across samples of an entire health system and did not examine how such metrics of use could be applied to patient portal use among unique patient populations.…”

“…Audit log files are an automatically produced, server-side documentation of user behavior within a particular system. 20 Originally designed for monitoring purposes, these files typically include four pieces of data: who accessed, which page, when, and the action they performed on that page. 21 While the exact granularity of the log files depends on the vendor's software, the use of this information allows researchers to unobtrusively study a wide range of user activities within a system, such as an electronic health record or an outpatient portal.…”

“…However, the scale at which this activity information exists has been harnessed to summarize sequences of activity and further identify groups of common sequential actions. 20 – 23 …”

Personas for Better Targeted eHealth Technologies: User-Centered Design Approach

“…Server log data are an automatic registration of, among others, the time, date, and activity that is carried out by the eHealth user within the system. An example is the identification of user groups based on activities within the eHealth system, resulting in personas characterized by activities that are most prominent within the clusters [ 14 ]. A more comprehensive approach is described in the study by Jones et al [ 15 ], in which activities within the system are expanded with information about the frequency, intensity, consistency, and demographics of the users.…”

Personas for Better Targeted eHealth Technologies: User-Centered Design Approach (Preprint)

“…Server log data are an automatic registration of, among others, the time, date, and activity that is carried out by the eHealth user within the system. An example is the identification of user groups on the basis of activities within the eHealth system, resulting in personas characterized by the activities that are most prominent within the clusters [14]. A more comprehensive approach is described in the study by Jones et al [15], in which activities within the system are expanded with information about the frequency, intensity, consistency and demographics of the users.…”