User-adaptive models for activity and emotion recognition using deep transfer learning and data augmentation

Garcia-Ceja, Enrique; Riegler, Michael; Kvernberg, Anders Kvalvik; Tørresen, Jim

doi:10.1007/s11257-019-09248-1

Cited by 23 publications

(18 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The efficiency of the proposed approach was compared with the existing approaches, and the results provided by the proposed study were better in terms of efficiency, accuracy, computational cost, and response time. Garcia-Ceja et al [39] designed predictive models to test user adaptive models that adapt to individual user behaviors and characteristics with condensed training data. The approaches were trained through data augmentation and deep transfer learning and were tested on two datasets; one is activity recognition dataset, and the other is emotion recombination dataset.…”

Section: Deep Learning and Transfer Learning Approaches For Health Monitoringmentioning

confidence: 99%

An Overview on Analyzing Deep Learning and Transfer Learning Approaches for Health Monitoring

Wang

Nazir

Shafiq

2021

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

With the rise and advancement of technology, early detection and involvement in health-associated monitoring through home control are growing with population aging. The expansion of healthy life expectations is progressively significant due to the speedy aging of the world population. The patient requires early and home-based treatment to detect and prevent disease on time and with less effort. Home-based health monitoring has been considered the need of a smart home. The services of health monitoring can facilitate the patient by collecting and analyzing the data of health for tackling diverse complex issues of health at a large scale. Health monitoring is a sustainable progression of clinical trials for ensuring that health is monitored according to the defined protocol and standard operating procedures. Various scenarios can be considered for monitoring health and are performed through experts of the field. Healthcare systems are having large-scale infrastructure of electronic devices, medical information systems, wearable and smart devices, medical records, and handheld devices. The growth in medical infrastructure, combined with the development of computational approaches in healthcare, has empowered practitioners and researchers to devise a novel solution in the innovative spectra. A detailed report of the existing literature in terms of deep learning and transfer learning is the dire need and facilitating of modern healthcare. To overcome these limitations, therefore, the proposed study presents a comprehensive review of the existing approaches, techniques, and methods associated with deep learning and transfer learning for health monitoring. This review will help researchers to formulate new ideas for facilitating healthcare based on the existing evidence.

show abstract

Section: Deep Learning and Transfer Learning Approaches For Health Monitoringmentioning

confidence: 99%

An Overview on Analyzing Deep Learning and Transfer Learning Approaches for Health Monitoring

Wang

Nazir

Shafiq

2021

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

show abstract

“…Different AI techniques have been studied throughout the following two decades to build human-interactive systems specifically for each particular user. In [25], the authors build user-adaptive models for activity and emotion recognition. These models are predictive models that adapt to each user's characteristics and behaviors with reduced training data.…”

Section: State Of the Artmentioning

confidence: 99%

Modeling and “smart” prototyping human-in-the-loop interactions for AmI environments

Gil

Albert

Fons

et al. 2021

Pers Ubiquit Comput

View full text Add to dashboard Cite

Autonomous capabilities are required in AmI environments in order to adapt systems to new environmental conditions and situations. However, keeping the human in the loop and in control of such systems is still necessary because of the diversity of systems, domains, environments, context situations, and social and legal constraints, which makes full autonomy a utopia within the short or medium term. Human-system integration introduces an important number of challenges and problems that have to be solved. On the one hand, humans should interact with systems even in those situations where their attentional, cognitive, and physical resources are limited in order to perform the interaction. On the other hand, systems must avoid overwhelming the user with unnecessary actions. Therefore, appropriate user-centered methods for AmI development should be used to help designers analyze and design human-in-the-loop interactions in AmI environments. This paper presents a user-centered design method that defines a process with a set of tools and techniques that supports the process steps in order to systematically design, prototype, and validate human-in-the-loop (HiL) solutions. The process starts with the definition of the HiL design, which defines how the system cooperates with the human. This HiL design is built using a conceptual framework that focuses on achieving human-system interactions that get human attention and avoid obtrusiveness. Then, we provide a software infrastructure to generate a prototype based on the HiL design and validate it by having end-users use a web simulator. The feedback data generated during the prototype user validation is gathered and used by a machine learning tool that infers the user’s needs and preferences. Finally, these inferences are used to automatically enhance the human-in-the-loop designs and prototypes. We have validated the proposed method through a twofold perspective: an experiment to analyze the perception of interaction designers regarding their acceptance of the design method and another experiment to evaluate the usefulness of the “smart” prototyping technique. The results obtained point out the acceptability of the proposed method by designers and the useful adaptations provided by the “smart” prototyping technique to achieve a HiL design that adapts well to users’ preferences and needs.

show abstract

“…[, ] = max [ ′ , − 1] ′ , ( ) In Table 1a, we compare user adaptive results from HMMs on all three pose estimation frameworks with results from Transformers. User adaptive refers to models initialized with a user independent model and updated with data from a target user [11]. To emulate this, we combine all data and perform stratifed 10-fold Cross Validation (CV) where test sets are not part of training sets.…”

Section: American Sign Language Recognitionmentioning

confidence: 99%

CopyCat: Using Sign Language Recognition to Help Deaf Children Acquire Language Skills

Bansal

Ravi

et al. 2021

Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Deaf children born to hearing parents lack continuous access to language, leading to weaker working memory compared to hearing children and deaf children born to Deaf parents. CopyCat is a game where children communicate with the computer via American Sign Language (ASL), and it has been shown to improve language skills and working memory. Previously, CopyCat depended on unscalable hardware such as custom gloves for sign verifcation, but modern 4K cameras and pose estimators present new opportunities. Before re-creating the CopyCat game for deaf children using of-the-shelf hardware, we evaluate whether current ASL recognition is sufcient.Using Hidden Markov Models (HMMs), user independent word accuracies were 90.6%, 90.5%, and 90.4% for AlphaPose, Kinect, and MediaPipe, respectively. Transformers, a state-of-the-art model in natural language processing, performed 17.0% worse on average.Given these results, we believe our current HMM-based recognizer can be successfully adapted to verify children's signing while playing CopyCat. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing systems and tools; Accessibility technologies; • Applied computing → Computer-managed instruction; • Computing methodologies → Machine learning; Feature selection.

show abstract

User-adaptive models for activity and emotion recognition using deep transfer learning and data augmentation

Cited by 23 publications

References 52 publications

An Overview on Analyzing Deep Learning and Transfer Learning Approaches for Health Monitoring

An Overview on Analyzing Deep Learning and Transfer Learning Approaches for Health Monitoring

Modeling and “smart” prototyping human-in-the-loop interactions for AmI environments

CopyCat: Using Sign Language Recognition to Help Deaf Children Acquire Language Skills

Contact Info

Product

Resources

About