LaplaceNet: A Hybrid Graph-Energy Neural Network for Deep Semisupervised Classification

Sellars, Philip; Avilés-Rivero, Angelica I.; Schönlieb, Carola-Bibiane

doi:10.1109/tnnls.2022.3203315

Cited by 11 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the second category, there are semi-supervised approaches, such as MixMatch [1], FixMatch [56] and FlexMatch [77], and others [1,41,48,52,69] that use pseudo-labeling or selflabeling, where unlabelled data is assigned pseudo-labels. Generative or teacher-student or ensemble models can also be listed among the semi-supervised approaches [12,16,23,33,38,49,53,55,59,64,65].…”

Section: Introductionmentioning

confidence: 99%

VCL-PL:Semi-Supervised Learning from Noisy Web Data with Variational Contrastive Learning

Yavuz

Yanıkoğlu

2022

2022 26th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

We address the problem of web supervised learning, in particular for face attribute classification. Web data suffers from image set noise, due to unrelated images that may be retrieved in response to the query. We propose a semi-supervised pseudo-labeling approach where the embedding space distribution is learnt via variational contrastive learning. We use 40 Gaussian sampling heads for the 40 attributes in the CelebA dataset and apply supervised contrastive learning over a limited amount of labelled data, to address the multi-label face attribute classification problem. Soft pseudo-labeling is then used to label the unlabelled data at attribute level, followed by two-stage domain adaptation. We show that the proposed method using noisy web data brings improvements in accuracy over supervised multi-label face attribute classification in all experimental settings (over 2% points for very low-data setting). We suggest that learning the embedding distribution and the subsequent soft pseudo-labeling according to the nearest neighbors help in overcoming the noise in the unlabeled data.

show abstract

Section: Introductionmentioning

confidence: 99%

VCL-PL:Semi-Supervised Learning from Noisy Web Data with Variational Contrastive Learning

Yavuz

Yanıkoğlu

2022

2022 26th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

show abstract

“…Another body of work [52][53][54][55] that is similar to the former mentioned works but focuses more on producing fake labels also known as pseudo labels that are eventually becoming accurate labelled data. Pseudo-labeling is one of the methods to implicitly minimize the class overlap entropy thus achieving low density separation [23,46].…”

Section: Semi-supervised Learningmentioning

confidence: 99%

“…Typically, their architectures involve a teacher that generates pseudo labels and a student that learns from the pseudo labels in an iterative way. However, there is also other work by [55] demonstrated effectiveness of self-training (doesn't involve teacher) data distillation [57] and graphical propagation [24] in generating pseudo labels strictly with data augmentation. According to [56], distillation is to transfer knowledge from the cumbersome model (teacher) to the distilled model (student) using the soft target (pseudo label) of a transfer set.…”

Section: Semi-supervised Learningmentioning

confidence: 99%

Deep learning application on head CT images

How

View full text Add to dashboard Cite

I studied on the deep learning frontier application on head Computed Tomography (CT) scans due to the wide adoption and cost efficiency of CT scans. The works were formulated in 4 chapters; In Chapter 1, I introduced the technical knowledge of CT images and the medical problems being solved using deep learning. In Chapter 2, I dived into the problem of image segmentation for brain intracranial hemorrhage (ICH) with small annotated mask dataset. My proposed training framework Meta Pseudo Segmentation (MPS) trained segmentation model with consistency training and student-teacher learning, outperforming supervised learning and EM algorithm. In Chapter 3, I tackled the problem of pseudo-healthy generation using VQGAN. My method outperformed the previous work substantially in synthesis quality. In Chapter 4, I proposed a unified multi-task segmentation model to perform ICH segmentation and brain tissue segmentation on CT. My model performed best in segmenting complex and granular region.

show abstract

Evaluating Hebbian Learning in a Semi-supervised Setting

Lagani

Falchi

Gennaro

et al. 2022

Machine Learning, Optimization, and Data Science

View full text Add to dashboard Cite

We propose a semi-supervised learning strategy for deep Convolutional Neural Networks (CNNs) in which an unsupervised pretraining stage, performed using biologically inspired Hebbian learning algorithms, is followed by supervised end-to-end backprop fine-tuning. We explored two Hebbian learning rules for the unsupervised pre-training stage: soft-Winner-Takes-All (soft-WTA) and nonlinear Hebbian Principal Component Analysis (HPCA). Our approach was applied in sample efficiency scenarios, where the amount of available labeled training samples is very limited, and unsupervised pre-training is therefore beneficial. We performed experiments on CIFAR10, CIFAR100, and Tiny ImageNet datasets. Our results show that Hebbian outperforms Variational Auto-Encoder (VAE) pre-training in almost all the cases, with HPCA generally performing better than soft-WTA.

show abstract

LaplaceNet: A Hybrid Graph-Energy Neural Network for Deep Semisupervised Classification

Cited by 11 publications

References 26 publications

VCL-PL:Semi-Supervised Learning from Noisy Web Data with Variational Contrastive Learning

VCL-PL:Semi-Supervised Learning from Noisy Web Data with Variational Contrastive Learning

Deep learning application on head CT images

Evaluating Hebbian Learning in a Semi-supervised Setting

Contact Info

Product

Resources

About