Discriminative Few Shot Learning of Facial Dynamics in Interview Videos for Autism Trait Classification

Zhang, Na; Ruan, Mindi; Wang, Shuo; Paul, Lynn; Li, Xin

doi:10.1109/taffc.2022.3178946

Cited by 13 publications

(3 citation statements)

References 83 publications

(147 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In medical image processing, due to the difficulty of biopsy label acquisition, Qinghua et al first attempted to introduce the few-shot learning into the ultrasound breast tumor diagnosis system [31] and achieved excellent performance. In recent years, the few-shot method has been widely used in the medical field, including the recognition of COVID-19 from rare chest images [32], human cell categorization in rare datasets [33], autism facial feature categorization [34], skin image categorization [35], and healthcare safety monitoring [36].…”

Section: B the Development Of Few/zero-shot Learning In Different Fieldsmentioning

confidence: 99%

Generative and Contrastive Combined Support Sample Synthesis Model for Few-/Zero-Shot Surface Defect Recognition

Dong,

Xie,

et al. 2024

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

Surface defect detection is one of the most important vision-based measurements for intelligent manufacturing. Existing detection methods mainly require massive numbers of defect samples to train the model to detect the defects. Nowadays, inadequate defect samples and labels are inevitably encountered in industrial data environments due to the highly automated and stable production lines escalatingly deployed, causing fewer and fewer defective products to be produced. Consequently, manual interventions are deeply required to analyze the abnormal sample once an unseen defect accidentally emerges that significantly decreases productivity. To this end, this paper proposes a novel few/zero-shot compatible surface defect detection method without requiring massive or even any defect samples to detect surface defects. First, a novel contrastive generator is proposed to use defects' text descriptions to synthesize "fake" visual features for those rare defects. Then, the synthesized visual features (for support samples) are fused with "real" visual features (for query samples) into a similarity graph to align the relationships between support samples and query samples. After, a class center optimization method is proposed to iteratively update the similarity matrix of the graph to obtain the classification probabilities for the query samples. Eventually, the proposed method solves the problem of the lack of defect samples and the inability of few-shot learning-based methods to recognize unseen classes. Massive experiments on eight fine-grained datasets show that our method gains an average of +8.29% improvements on few-shot recognition tasks and achieves an average of +8.23% improvements on zero-shot recognition tasks compared with the state-of-the-art method. Moreover, the proposed method is deployed in a real-world prototype system, and the method's feasibility is finally demonstrated. The core code of the proposed method is available at: https://github.com/NDYBSNDY/AsC

show abstract

Section: B the Development Of Few/zero-shot Learning In Different Fieldsmentioning

confidence: 99%

Generative and Contrastive Combined Support Sample Synthesis Model for Few-/Zero-Shot Surface Defect Recognition

Dong,

Xie,

et al. 2024

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

show abstract

“…A set of Long-Short Term Memory (LSTM) models were then utilised for classification. Another study used videos from ADOS interviews to extract spatiotemporal facial features for classification [27]. In non-ASD literature, an appearance-based feature called the Gait Energy Image (GEI) [28], that encapsulates spatiotemporal characteristics of gait in a single 2D image, has been commonly paired with the Convolutional Neural Network (CNN) [29] in general gait-based classification problems.…”

Section: Appearance-based Featuresmentioning

confidence: 99%

Encoding Kinematic and Temporal Gait Data in an Appearance-Based Feature for the Automatic Classification of Autism Spectrum Disorder

Henderson,

Yogarajah,

Gardiner

et al. 2023

IEEE Access

View full text Add to dashboard Cite

In appearance-based gait analysis studies, Gait Energy Images (GEI) have been shown to be an effective tool for human identification and gait pathology detection. In addition, model-based studies found kinematic and spatio-temporal features to be useful for gait recognition and Autism Spectrum Disorder (ASD) classification. Adapting the GEI to focus on the strong ASD features would improve the early screening of ASD by allowing the use of powerful appearance-based classifiers such as Convolutional Neural Networks (CNN). This paper introduces an enhanced GEI, by averaging images from a video sequence to produce a single image but by retention of a person's joint positions only, instead of the full body silhouettes. Depth is encoded into the binary images before they are averaged using colour mapping, a technique used in the Chrono-Gait Image. The Joint Energy Image (JEI) therefore embeds both the temporal and depth information of the joints into a 2D image. The image was preprocessed using Principal Component Analysis before being applied to a Multi-Layer Perceptron, and a Random Forest classifier. The JEI was also applied to a CNN directly. Accuracy was improved by using a Test Time Augmentation (TTA) measure where the original and augmented JEIs were used in a voting system to obtain a prediction. The CNN achieved a TTA accuracy of 95.56% when trained on a primary dataset of 100 subjects (50 with ASD and 50 that are typically developed), and 80% TTA accuracy on a secondary dataset of 20 subjects (10 ASD and 10 typically developed) across multiple tests, thus showing the viability of both the CNN and the JEI to compete with state-of-the-art feature sets and performances.

show abstract

“…Experiments are conducted on synthetic data set and some public face datasets, which show that the proposed PMFA is an effective method. As an effective feature extraction method, like MFA and its variants, PMFA can be applied in many fields, such as face recognition [13][16] facial expression recognition [22], autism trait classification [23], image representation [24], etc.…”

Section: Introductionmentioning

confidence: 99%

Marginal Fisher Analysis With Polynomial Matrix Function

Ran

Wang

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Marginal fisher analysis (MFA) is a dimensionality reduction method based on a graph embedding framework. In contrast to traditional linear discriminant analysis (LDA), which requires the data to follow a Gaussian distribution, MFA is suitable for non-Gaussian data, and it has better pattern classification ability. However, MFA has the small-sample-size (SSS) problem. This paper aims to solve the small-sample-size problem while increasing the classification performance of MFA. Based on a matrix function dimensionality reduction framework, the criterion of the MFA method is reconstructed by using the polynomials matrix function transformation, and then a new MFA method is proposed, named PMFA (polynomial marginal fisher analysis). The major contributions of the proposed PMFA method are that it solves the small-sample-size problem of MFA, and it can enlarge the distance between marginal sample points of inter-class, so that it can get better pattern classification performance. Experiments on the public face datasets show that PMFA can get a better classification ability than MFA and its improved methods.INDEX TERMS Dimensionality reduction, manifold learning, marginal fisher analysis, matrix function, the small-sample-size (SSS) problem.

show abstract

Discriminative Few Shot Learning of Facial Dynamics in Interview Videos for Autism Trait Classification

Cited by 13 publications

References 83 publications

Generative and Contrastive Combined Support Sample Synthesis Model for Few-/Zero-Shot Surface Defect Recognition

Generative and Contrastive Combined Support Sample Synthesis Model for Few-/Zero-Shot Surface Defect Recognition

Encoding Kinematic and Temporal Gait Data in an Appearance-Based Feature for the Automatic Classification of Autism Spectrum Disorder

Marginal Fisher Analysis With Polynomial Matrix Function

Contact Info

Product

Resources

About