Head Motion Tracking Through in-Ear Wearables

Ferlini, Andrea; Montanari, A.; Mascolo, Cecilia; Harle, Robert

doi:10.1145/3345615.3361131

Cited by 37 publications

(11 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, their study has shown that great variability exists in how people wear the sensor in the ear (i.e., different angle/orientations), which further complicates estimating head pose between different subjects. While it is also possible to estimate head orientation using only wireless wearable sensors (e.g., smart ear pieces and head bands) [18,23], these technology are still in nascent stages and are not able to estimate orientations accurately. A recent technology based on near-infrared sensing utilizes pairwise light sensing to infer the incident angle and distance between two sensors [41].…”

Section: Wearable Sensor Based Orientation Measurement and Estimationmentioning

confidence: 99%

Multimodal Joint Head Orientation Estimation in Interacting Groups via Proxemics and Interaction Dynamics

Tan

Tax

Hung

2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

Human head orientation estimation has been of interest because head orientation serves as a cue to directed social attention. Most existing approaches rely on visual and high-fidelity sensor inputs and deep learning strategies that do not consider the social context of unstructured and crowded mingling scenarios. We show that alternative inputs, like speaking status, body location, orientation, and acceleration contribute towards head orientation estimation. These are especially useful in crowded and in-the-wild settings where visual features are either uninformative due to occlusions or prohibitive to acquire due to physical space limitations and concerns of ecological validity. We argue that head orientation estimation in such social settings needs to account for the physically evolving interaction space formed by all the individuals in the group. To this end, we propose an LSTM-based head orientation estimation method that combines the hidden representations of the group members. Our framework jointly predicts head orientations of all group members and is applicable to groups of different sizes. We explain the contribution of different modalities to model performance in head orientation estimation. The proposed model outperforms baseline methods that do not explicitly consider the group context, and generalizes to an unseen dataset from a different social event.

show abstract

Section: Wearable Sensor Based Orientation Measurement and Estimationmentioning

confidence: 99%

Multimodal Joint Head Orientation Estimation in Interacting Groups via Proxemics and Interaction Dynamics

Tan

Tax

Hung

2021

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

show abstract

“…We can see that the signals for the same activity show similar patterns under both cases, indicating head movements have negligible impact on the sensing signals. 4 In addition, as shown in Figure 5, background noise has no impact on the sensing signals as it is naturally suppressed with the occlusion of human ear. Moreover, as the target motion signals are in frequencies below 50 Hz, any audible background noise can be easily removed with a low-pass filter.…”

Section: Initial Explorationmentioning

confidence: 99%

OESense

Ferlini

Mascolo

2021

Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services

Self Cite

View full text Add to dashboard Cite

Smart earbuds are recognized as a new wearable platform for personal-scale human motion sensing. However, due to the interference from head movement or background noise, commonly-used modalities (e.g. accelerometer and microphone) fail to reliably detect both intense and light motions. To obviate this, we propose OESense, an acoustic-based in-ear system for general human motion sensing. The core idea behind OESense is the joint use of the occlusion effect (i.e., the enhancement of low-frequency components of bone-conducted sounds in an occluded ear canal) and inward-facing microphone, which naturally boosts the sensing signal and suppresses external interference. We prototype OESense as an earbud and evaluate its performance on three representative applications, i.e., step counting, activity recognition, and hand-to-face gesture interaction. With data collected from 31 subjects, we show that OESense achieves 99.3% step counting recall, 98.3% recognition recall for 5 activities, and 97.0% recall for five tapping gestures on human face, respectively. We also demonstrate that OESense is compatible with earbuds' fundamental functionalities (e.g. music playback and phone calls). In terms of energy, OESense consumes 746 mW during data recording and recognition and it has a response latency of 40.85 ms for gesture recognition. Our analysis indicates such overhead is acceptable and OESense is potential to be integrated into future earbuds. CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile computing systems and tools.

show abstract

“…Activity modeling: We can recognise upper body movements, such as head and neck activities [6,21], facial activities and expression [18], and whole-body movements, i.e., walking, standing, falling, etc. [21].…”

Section: Earable As Memory Aidmentioning

confidence: 99%

Designing Memory Aids for Dementia Patients using Earables

Franklin¹,

Lagnado²,

Min

et al. 2021

Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of The

View full text Add to dashboard Cite

Globally around 50 million people are currently living with dementia, and there are nearly 10 million new cases every year. The decline of memory and, with it, lack of self-confidence and continuous confusion have a devastating effect on people living with this disease. Dementia patients even struggle to accomplish mundane chores and require assistance for daily living and social connectedness. Over the past decade, we have seen remarkable growth in wearable technologies to manage our health and wellbeing and improve our awareness and social connectedness. However, we have to ask why wearables are not addressing this fundamental challenge of memory augmentation that threatens our society? Some limited existing work on cognitive wearables for dementia has focused on using images via camera-based life-logging technology. Instead, in this paper, we argue that earable -by virtue of its unique placement, rich sensing modalities, and acoustic feedback capabilities, uncovers new opportunities to augment human cognition to address this pressing need to assist dementia patients. To this end, we delve into fundamental principles of cognitive neuroscience to understand what constitutes memory disorder and its symptoms concerning errors in everyday activities. Building on this, we discuss the benefits of earables (in conjunction with smart objects) in modelling activity and intention of dementia patients and providing contextual memory cues. We put forward a guidance system to assist dementia patients with daily living and social connectedness. CCS CONCEPTS• Computer systems organization → Embedded systems.

show abstract

Head Motion Tracking Through in-Ear Wearables

Cited by 37 publications

References 13 publications

Multimodal Joint Head Orientation Estimation in Interacting Groups via Proxemics and Interaction Dynamics

Multimodal Joint Head Orientation Estimation in Interacting Groups via Proxemics and Interaction Dynamics

OESense

Designing Memory Aids for Dementia Patients using Earables

Contact Info

Product

Resources

About