Feature Integration Through Semi-Supervised Multimodal Gaussian Process Latent Variable Model With Pseudo-Labels for Interest Level Estimation

Abstract

This study presents a novel feature integration method for interest level estimation using a semi-supervised multimodal Gaussian process latent variable model with pseudo-labels (semi-MGPPL). Semi-MGPPL is an extended version of the multimodal Gaussian process latent variable model (mGPLVM). It integrates features calculated from multiple modalities to predict the users' interest levels in content. It is known that reflecting known interest levels of known users in the latent space effectively improves the accuracy of interest level estimation. However, previous methods have difficulty reflecting the interest levels when the number of samples is insufficient. Semi-MGPPL efficiently reflects interest levels in the latent space by pseudo-labeling of unlabeled samples and increasing the number of available pairs among labeled samples. In addition, obtaining behavior features is difficult for a new test sample. However, requirement of features of all modalities by previous mGPLVM-based methods makes the calculation of latent variables of a test sample challenging. Semi-MGPPL solves this problem by training a projection function from the original feature to the latent space. The experimental results on real data demonstrate the effectiveness and robustness of semi-MGPPL.INDEX TERMS Gaussian process, multimodal analysis, feature integration, semi-supervised learning, pseudo-label.