Toward Adaptive Information Fusion in Multimodal Systems

Oviatt, Sharon

doi:10.1007/11677482_2

Cited by 11 publications

(2 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Integration patterns are also investigated in [3,4], were machine learning based approaches to predict a user's integration pattern were presented, based on the data of [7]. It is shown that 15 samples per user are enough to predict a user's integration pattern with an accuracy of 81%, so integration patterns can be detected relatively fast, even in an automated way.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Interaction History and its use in Error Detection and Recovery

Schüssel

Honold

Schmidt

et al. 2014

Proceedings of the 16th International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Multimodal systems still tend to ignore the individual input behavior of users, and at the same time, suffer from erroneous sensor inputs. Although many researchers have described user behavior in specific settings and tasks, little to nothing is known about the applicability of such information, when it comes to increase the robustness of a system for multimodal inputs. We conducted a gamified experimental study to investigate individual user behavior and error types found in an actually running system. It is shown, that previous ways of describing input behavior by a simple classification scheme (like simultaneous and sequential) are not suited to build up an individual interaction history. Instead, we propose to use temporal distributions of different metrics derived from multimodal event timings. We identify the major errors that can occur in multimodal interactions and finally show how such an interaction history can practically be applied for error detection and recovery. Applying the proposed approach to the experimental data, the initial error rate is reduced from 4.9% to a minimum of 1.2%.

show abstract

Section: Related Workmentioning

confidence: 99%

Multimodal Interaction History and its use in Error Detection and Recovery

Schüssel

Honold

Schmidt

et al. 2014

Proceedings of the 16th International Conference on Multimodal Interaction

View full text Add to dashboard Cite

show abstract

“…A novel trend of work investigates explicit integration of machine learning algorithms into the data collection process to accomplish adaptation. For example, machine learning methods are deployed in [36] to achieve on-line adaptation to users' multimodal temporal thresholds within a human computer interaction application framework. Some other work studies application of reinforcement learning to adaptive fusion systems to perform dynamic data reliability estimation [5,35].…”

Section: Related Workmentioning

confidence: 99%

Sleep behavior assessment via smartwatch and stigmergic receptive fields

Alfeo

Barsocchi

Cimino

et al. 2017

Pers Ubiquit Comput

View full text Add to dashboard Cite

Sleep behavior is a key factor in maintaining good physiological and psychological health. A well-known approach to monitor sleep is polysomnography. However, it is costly and intrusive, which may disturb sleep. Consequently, polysomnography is not suitable for sleep behavior analysis. Other approaches are based on actigraphy and sleep diary. Although being a good source of information for sleep quality assessment, sleep diaries can be affected by cognitive bias related to subject’s sleep perception, while actigraphy overestimates sleep periods and night-time disturbance compared to sleep diaries. Machine learning techniques can improve the objectivity and reliability of the observations. However, since signal morphology vary widely between people, conventional machine learning is complex to set up. In this regard, we present an adaptive, reliable, and innovative computational approach to provide per-night assessment of sleep behavior to the end-user. We exploit heartbeat rate and wrist acceleration data, gathered via smartwatch, in order to identify subject’s sleep behavioral pattern. More specifically, heartbeat rate and wrist motion samples are processed via computational stigmergy, a bio-inspired scalar and temporal aggregation of samples. Stigmergy associates each sample to a digital pheromone deposit (mark) defined in a mono-dimensional space and characterized by evaporation over time. As a consequence, samples close in terms of time and intensity are aggregated into functional structures called trails. The stigmergic trails allow to compute the similarity between time series on different temporal scales, to support classification or clustering processes. The overall computing schema includes a parametric optimization for adapting the structural parameters to individual sleep dynamics. The outcome is a similarity between sleep nights of the same subject, to generate clusters of nights with different quality levels. Experimental results are shown for three real-world subjects. The resulting similarity is also compared with the dynamic time warping, a popular similarity measure for time series

show abstract