Understanding practices and needs of researchers in human state modeling by passive mobile sensing

Abstract

Passive mobile sensing for the purpose of human state modeling is a fast-growing area. It has been applied to solve a wide range of behavior-related problems, including physical and mental health monitoring, affective computing, activity recognition, routine modeling, etc. However, in spite of the emerging literature that has investigated a wide range of application scenarios, there is little work focusing on the lessons learned by researchers, and on guidance for researchers to this approach. How do researchers conduct these types of research studies? Is there any established common practice when applying mobile sensing across different application areas? What are the pain points and needs that they frequently encounter? Answering these questions is an important step in the maturing of this growing sub-field of ubiquitous computing, and can benefit a wide range of audiences. It can serve to educate researchers who have growing interests in this area but have little to no previous experience. Intermediate researchers may also find the results interesting and helpful for reference to improve their skills. Moreover, it can further shed light on the design guidelines for a future toolkit that could facilitate research processes being used. In this paper, we fill this gap and answer these questions by conducting semi-structured interviews with ten experienced researchers from four countries to understand their practices and pain points when conducting their research. Our results reveal a common pipeline that researchers have adopted, and identify major challenges that do not appear in published work but that researchers often encounter. Based on the results of our interviews, we discuss practical suggestions for novice researchers and high-level design principles for a toolkit that can accelerate passive mobile sensing research.

Self-supervised Pretraining and Transfer Learning Enable Flu and COVID-19 Predictions in Small Mobile Sensing Datasets

“…As previously mentioned, the majority of prior work in behavioral modeling has focused on classification tasks with hand-crafted features. While neural minutelevel models may achieve superior performance than simple classifiers trained on these features, there is nonetheless a large body of work supporting the utility of handcrafted features in sensing [7,17,31,35,40,43,48,66,70]. For this pretraining task, we ask the model to perform a multiple regression to predict the daily features in Table 2 on the final day of the seven day window (Figure 1C).…”

“…• XGBoost: How well does our model perform relative to a non-neural baseline? Boosted decision trees are frequently used in many sensing studies because they are supported by common, easy to use libraries and often achieve strong performance out-of-the-box [66]. Since boosted trees expectedly do not scale well to the thousands of observations in our raw time series data, we compute a set of commonly used features for each day in the window, and then concatenate these features for a final input.…”

“…Labeled behavioral data is often prohibitively expensive to collect, particularly in the context of public health where ground-truth labels require costly testing infrastructure and study management. Accordingly, many studies from prior work operate on data with on the order of dozen participants [66]. In this regard, one promising application of generalizable self-supervised pretraining is that models could leverage large unlabeled datasets to improve predictive power in settings with limited labeled training data.…”

“…While neural representation learning approaches have provided transformative performance improvements across Natural Language Processing (NLP) and Computer Vision (CV) [13,20,29,30,33,37,46], currently these techniques are rarely adopted for mobile sensing research and applications. In contrast to typical NLP and CV benchmark datasets, mobile sensing data are usually very limited in size (often less than 20 individuals due to arduous and expensive data collection [66]), frequently contain missing data (93% of days in our data; e.g. when device is not used or charging), consist of long history time series (>10,000 min-by-min time steps for one week of data) with relevant long range dependencies (e.g.…”

“…because most people are not sick on most days) [66]. Therefore, researchers have often been limited to using small datasets and less data hungry non-neural models such as boosted tree models with hand-crafted features which typically perform worse (e.g., [7,17,31,35,40,43,48,66,70]).…”