Smartphone-derived Virtual Keyboard Dynamics Coupled with Accelerometer Data as a Window into Understanding Brain Health

Ning, Emma; Cladek, Andrea T.; Ross, Mindy K.; Kabir, Sarah; Barve, Amruta; Kennelly, Ellyn; Hussain, Faraz; Duffecy, Jennifer; Langenecker, Scott A.; Nguyen, Theresa; Tulabandhula, Theja; Zulueta, John; Ajilore, Olusola; Demos, Alexander P.; Leow, Alex D.

doi:10.1145/3544548.3580906

Cited by 10 publications

(18 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The BiAffect preprocessing pipeline was based on previous studies. 8,14 In brief, all keyboard and accelerometery data were aggregated to the daily level. We extracted the following features: 1) median inter-key delay (IKD), an inverse measure of typing speed, 2) 95 th percentile IKD, a measure of pausing within typing sessions, 3) mean absolute deviation (MAD) IKD, which quantifies typing speed variability, 8 4) autocorrect rate, 5) backspace rate, 6) the total number of key presses per day, 7) the percentage of typing sessions spent upright, and 8) the percentage of typing sessions where the phone recorded movement.…”

Section: Resultsmentioning

confidence: 99%

“…Accelerometer data were grouped into typing sessions and low-pass filtered using a second-order bidirectional Butterworth filter with a cutoff frequency of 4 Hz to remove noise. 14…”

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

“…Accelerometer data were grouped into typing sessions and low-pass filtered using a secondorder bidirectional Butterworth filter with a cutoff frequency of 4 Hz to remove noise. 14 Afterwards, every sample within a session was classified as either moving (also 'active') or stationary based on the magnitude of the filtered x, y, and z accelerometer readings. Magnitudes close to 1 reflect the natural gravitational pull of the earth and therefore indicate the user's phone is at rest.…”

Section: Accelerometer Data Preprocessingmentioning

confidence: 99%

“…If the median z value of a session was (strictly) below 0.1 and the median x value was in-between -0.2 and 0.2 (inclusive), the session was classified as upright. 14 Sessions that were not classified as upright could potentially indicate that participants were using their phone while lying down. Finally, by counting the number of active and upright sessions within a day and dividing those counts by the total number of sessions within a day, we get a rate of active and a rate of upright sessions per day.…”

Section: Accelerometer Data Preprocessingmentioning

confidence: 99%

See 4 more Smart Citations

Smartphone keyboard dynamics predict affect in suicidal ideation

Knol,

Nagpal,

Leaning

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

While digital phenotyping provides opportunities for unobtrusive, real-time mental health assessments, the integration of its modalities is not trivial due to high dimensionalities and discrepancies in sampling frequencies. We provide an integrated pipeline that solves these issues by transforming all modalities to the same time unit, applying temporal independent component analysis (ICA) to high-dimensional modalities, and fusing the modalities with linear mixed-effects models. We applied our approach to integrate high-quality, daily self-report data with BiAffect keyboard dynamics derived from a clinical suicidality sample of mental health outpatients. Applying the ICA to the self-report data (104 participants, 5712 days of data) revealed components related to well-being, anhedonia, and irritability and social dysfunction. Mixed-effects models (55 participants, 1794 days) showed that less phone movement while typing was associated with more anhedonia (β = -0.12, p = 0.00030). We consider this method to be widely applicable to dense, longitudinal digital phenotyping data.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Accelerometer data were grouped into typing sessions and low-pass filtered using a second-order bidirectional Butterworth filter with a cutoff frequency of 4 Hz to remove noise. 14…”

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Accelerometer Data Preprocessingmentioning

confidence: 99%

Section: Accelerometer Data Preprocessingmentioning

confidence: 99%

See 3 more Smart Citations

Smartphone keyboard dynamics predict affect in suicidal ideation

Knol,

Nagpal,

Leaning

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Smartphone keyboard dynamics predict affect in suicidal ideation

Knol,

Nagpal,

Leaning

et al. 2024

npj Digit. Med.

Self Cite

View full text Add to dashboard Cite

While digital phenotyping provides opportunities for unobtrusive, real-time mental health assessments, the integration of its modalities is not trivial due to high dimensionalities and discrepancies in sampling frequencies. We provide an integrated pipeline that solves these issues by transforming all modalities to the same time unit, applying temporal independent component analysis (ICA) to high-dimensional modalities, and fusing the modalities with linear mixed-effects models. We applied our approach to integrate high-quality, daily self-report data with BiAffect keyboard dynamics derived from a clinical suicidality sample of mental health outpatients. Applying the ICA to the self-report data (104 participants, 5712 days of data) revealed components related to well-being, anhedonia, and irritability and social dysfunction. Mixed-effects models (55 participants, 1794 days) showed that less phone movement while typing was associated with more anhedonia (β = −0.12, p = 0.00030). We consider this method to be widely applicable to dense, longitudinal digital phenotyping data.

show abstract

Toward Cognition-Aware Digital Phenotyping for Substance Use Disorder

Kong,

Singh

2023

Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing &Amp; The 2023 ACM Int

View full text Add to dashboard Cite

Smartphone-derived Virtual Keyboard Dynamics Coupled with Accelerometer Data as a Window into Understanding Brain Health

Cited by 10 publications

References 55 publications

Smartphone keyboard dynamics predict affect in suicidal ideation

Smartphone keyboard dynamics predict affect in suicidal ideation

Smartphone keyboard dynamics predict affect in suicidal ideation

Toward Cognition-Aware Digital Phenotyping for Substance Use Disorder

Contact Info

Product

Resources

About