Discovery of everyday human activities from long-term visual behaviour using topic models

Steil, Julian; Bulling, Andreas

doi:10.1145/2750858.2807520

Cited by 59 publications

(38 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other works have shown that eye movements are closely linked to mental disorders, such as Alzheimer's [Hutton et al 1984], Parkinson's [Kuechenmeister et al 1977], or schizophrenia [Holzman et al 1974]. More recent work in HCI has demonstrated the use of eye movement analysis for human activity recognition [Bulling et al 2013;Steil and Bulling 2015] as well as to infer a user's cognitive state [Bulling and Zander 2014;Faber et al 2017] or personality traits [Hoppe et al 2018]. More closely related to our work, several researchers have shown that gender and age can be inferred from eye movements, e.g.…”

Section: Information Available In Eye Movementsmentioning

confidence: 99%

Privacy-aware eye tracking using differential privacy

Steil

Hagestedt

Huang

et al. 2019

Proceedings of the 11th ACM Symposium on Eye Tracking Research &Amp; Applications

Self Cite

View full text Add to dashboard Cite

With eye tracking being increasingly integrated into virtual and augmented reality (VR/AR) head-mounted displays, preserving users' privacy is an ever more important, yet under-explored, topic in the eye tracking community. We report a large-scale online survey (N=124) on privacy aspects of eye tracking that provides the first comprehensive account of with whom, for which services, and to what extent users are willing to share their gaze data. Using these insights, we design a privacy-aware VR interface that uses differential privacy, which we evaluate on a new 20-participant dataset for two privacy sensitive tasks: We show that our method can prevent user re-identification and protect gender information while maintaining high performance for gaze-based document type classification. Our results highlight the privacy challenges particular to gaze data and demonstrate that differential privacy is a potential means to address them. Thus, this paper lays important foundations for future research on privacy-aware gaze interfaces. CCS CONCEPTS• Security and privacy → Human and societal aspects of security and privacy; • Human-centered computing → Human computer interaction (HCI);

show abstract

Section: Information Available In Eye Movementsmentioning

confidence: 99%

Privacy-aware eye tracking using differential privacy

Steil

Hagestedt

Huang

et al. 2019

Proceedings of the 11th ACM Symposium on Eye Tracking Research &Amp; Applications

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, gaze information has also shown significant potential for user understanding. Most intuitively, eye tracking techniques have been used to capture and infer user behaviours, such as eye contact [70] and daily activities [6,55]. Eye tracking data has been also used to recognise users' latent states, including interest and engagement [32,34], affective states [41], cognitive states [20,38], and attentive states [14,62].…”

Section: Gaze-based Human-computer Interactionmentioning

confidence: 99%

“…Gaze-based user modelling and passive eye monitoring require gaze estimation to detect gaze patterns instead of individual points. These gaze patterns could be large regions on the screen, such as during saliency prediction [66]; relative eye movements for inferring everyday activities [51,55], cognitive load and processes [7,59], or mobile interaction [60]; or off-line user behaviour analysis for game play [43]. For attentive user interfaces, usually it is sufficient to detect the attention of the user [1] with binary eye contact detection [12,42,54,70].…”

Section: Gaze Applicationsmentioning

confidence: 99%

“…Performance consistency between indoor and outdoor environments becomes more and more important with the popularisation of personal mobile devices. It has impacts on passive eye monitoring since usually longterm recording through daily life is necessary [55]. Explicit eye input and gaze-based user modelling also require such consistency as users could run these applications anywhere with their mobile devices.…”

Section: Gaze Applicationsmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications

Zhang¹,

Sugano

Bulling

2019

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Self Cite

100

View full text Add to dashboard Cite

Appearance-based gaze estimation methods that only require an off-the-shelf camera have significantly improved but they are still not yet widely used in the human-computer interaction (HCI) community. This is partly because it remains unclear how they perform compared to model-based approaches as well as dominant, special-purpose eye tracking equipment. To address this limitation, we evaluate the performance of state-of-the-art appearance-based gaze estimation for interaction scenarios with and without personal calibration, indoors and outdoors, for different sensing distances, as well as for users with and without glasses. We discuss the obtained findings and their implications for the most important gaze-based applications, namely explicit eye input, attentive user interfaces, gaze-based user modelling, and passive eye monitoring. To democratise the use of appearance-based gaze estimation and interaction in HCI, we finally present OpenGaze (www.opengaze.org), the first software toolkit for appearance-based gaze estimation and interaction. CCS CONCEPTS• Computing methodologies → Computer vision.

show abstract

“…These features include numerical features, such as pixel counts of semantic segmentations, entropy of objectness maps, and mean depth map values, as well as binary encodings like occurrence of a touch event or whether an application on the handheld device is active. We aggregate features over a window by computing the mean, maximum, minimum, standard deviation and slope for numerical features, and the mean and the slope for [4,5,41]. It is therefore conceivable that gaze features may help to improve the performance of our method for attention forecasting.…”

Section: Feature Extractionmentioning

confidence: 99%

Forecasting user attention during everyday mobile interactions using device-integrated and wearable sensors

Steil

Müller

Sugano

et al. 2018

Proceedings of the 20th International Conference on Human-Computer Interaction With Mobile Devices and Services

Self Cite

View full text Add to dashboard Cite

Visual attention is highly fragmented during mobile interactions, but the erratic nature of attention shifts currently limits attentive user interfaces to adapting after the fact, i.e. after shifts have already happened. We instead study attention forecasting -the challenging task of predicting users' gaze behaviour (overt visual attention) in the near future. We present a novel long-term dataset of everyday mobile phone interactions, continuously recorded from 20 participants engaged in common activities on a university campus over 4.5 hours each (more than 90 hours in total). We propose a proof-of-concept method that uses device-integrated sensors and body-worn cameras to encode rich information on device usage and users' visual scene. We demonstrate that our method can forecast bidirectional attention shifts and predict whether the primary attentional focus is on the handheld mobile device. We study the impact of different feature sets on performance and discuss the significant potential but also remaining challenges of forecasting user attention during mobile interactions.

show abstract

Discovery of everyday human activities from long-term visual behaviour using topic models

Cited by 59 publications

References 36 publications

Privacy-aware eye tracking using differential privacy

Privacy-aware eye tracking using differential privacy

Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications

Forecasting user attention during everyday mobile interactions using device-integrated and wearable sensors

Contact Info

Product

Resources

About