Learning signer-invariant representations with adversarial training

Ferreira, Pedro M.; Pernes, Diogo; Rebelo, Ana; Cardoso, Jaime S.

doi:10.1117/12.2559534

Cited by 3 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Adversarial Learning method [2] uses an additional network that learns to classify the different actors in the training set and later uses that knowledge to remove actor information from the features of the network, making it less biased toward the actors.…”

Section: Methodologiesmentioning

confidence: 99%

“…Another proposed method was to use adversarial losses for different scene types to mitigate scene biases [29]. Similarly, adversarial learning procedures allowed learning signer-invariant latent representations to be highly discriminating for sign recognition [2].…”

Section: B Representation Biasesmentioning

confidence: 99%

“…To accurately predict violence and non-violence in video, independently of the actors that are present, the model should be trained in a way where the latent representations learned by the model preserve the information relative to the action, and discard the information relative to the actors, which may negatively impact the classification. To accomplish this, the methodology proposed in [2] was adapted. It presents a network architecture and an adversarial training objective, which addresses the signer-independent problem (actorindependent problem, in our case).…”

Section: B Adversarial Learningmentioning

confidence: 99%

“…Dataset availability and size can be critical when dealing with individuals engaged in different activities, both for legal reasons and because of the effort involved in preparation. If the dataset used to train a model contains only a small number of actors, the model can be biased toward them, i.e., to specific individuals and their unique characteristics [2]. This causes the model to perform poorly when confronted with a different set of actors performing the same activities.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Toward Vehicle Occupant-Invariant Models for Activity Characterization

Capozzi

Barbosa

Pinto³

et al. 2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

With the advent of self-driving cars and the push by large companies into fully driverless transportation services, monitoring passenger behaviour in vehicles is becoming increasingly important for several reasons, such as ensuring safety and comfort. Although several human action recognition (HAR) methods have been proposed, developing a true HAR system remains a very challenging task. If the dataset used to train a model contains a small number of actors, the model can become biased towards these actors and their unique characteristics. This can cause the model to generalise poorly when confronted with new actors performing the same actions. This limitation is particularly acute when developing models to characterise the activities of vehicle occupants, for which data sets are short and scarce. In this study, we describe and evaluate three different methods that aim to address this actor bias and assess their performance in detecting in-vehicle violence. These methods work by removing specific information about the actor from the model's features during training or by using data that is independent of the actor, such as information about body posture. The experimental results show improvements over the baseline model when evaluated with real data.

show abstract

Section: Methodologiesmentioning

confidence: 99%

Section: B Representation Biasesmentioning

confidence: 99%

Section: B Adversarial Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Toward Vehicle Occupant-Invariant Models for Activity Characterization

Capozzi

Barbosa

Pinto³

et al. 2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…Second, we introduce an additional term to the adversarial training objective that further discourages the learned representations of retaining any signer-specific information, by explicitly imposing similarity in the latent distributions of different signers. This paper is an extension of our conference paper [4]. The new contributions of this paper are summarized as follows:…”

Section: Introductionmentioning

confidence: 95%

Signer-Independent Sign Language Recognition with Adversarial Neural Networks

Ferreira¹,

Multimedia²,

Pernes³

et al. 2021

IJMLC

Self Cite

View full text Add to dashboard Cite

Sign Language Recognition (SLR) has become an appealing topic in modern societies because such technology can ideally be used to bridge the gap between deaf and hearing people. Although important steps have been made towards the development of real-world SLR systems, signer-independent SLR is still one of the bottleneck problems of this research field. In this regard, we propose a deep neural network along with an adversarial training objective, specifically designed to address the signer-independent problem. Specifically, the proposed model consists of an encoder, mapping from input images to latent representations, and two classifiers operating on these underlying representations: (i) the sign-classifier, for predicting the class/sign labels, and (ii) the signer-classifier, for predicting their signer identities. During the learning stage, the encoder is simultaneously trained to help the sign-classifier as much as possible while trying to fool the signer-classifier. This adversarial training procedure allows learning signer-invariant latent representations that are in fact highly discriminative for sign recognition. Experimental results demonstrate the effectiveness of the proposed model and its capability of dealing with the large inter-signer variations.

show abstract

Condition Invariance for Autonomous Driving by Adversarial Learning

e Silva,

Cruz

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Learning signer-invariant representations with adversarial training

Cited by 3 publications

References 0 publications

Toward Vehicle Occupant-Invariant Models for Activity Characterization

Toward Vehicle Occupant-Invariant Models for Activity Characterization

Signer-Independent Sign Language Recognition with Adversarial Neural Networks

Condition Invariance for Autonomous Driving by Adversarial Learning

Contact Info

Product

Resources

About