Variational Autoencoder with Embedded Student-$t$ Mixture Model for Authorship Attribution

Abstract

Traditional computational authorship attribution describes a classification task in a closed-set scenario. Given a finite set of candidate authors and corresponding labeled texts, the objective is to determine which of the authors has written another set of anonymous or disputed texts. In this work, we propose a probabilistic autoencoding framework to deal with this supervised classification task. More precisely, we are extending a variational autoencoder (VAE) with embedded Gaussian mixture model to a Student-t mixture model. Autoencoders have had tremendous success in learning latent representations. However, existing VAEs are currently still bound by limitations imposed by the assumed Gaussianity of the underlying probability distributions in the latent space. In this work, we are extending the Gaussian model for the VAE to a Studentt model, which allows for an independent control of the "heaviness" of the respective tails of the implied probability densities. Experiments over an Amazon review dataset indicate superior performance of the proposed method.

Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows

“…To position our work in context, note that the encoder q(z | x) may be viewed as a variational approximation to the posterior p(z | x) defined by the decoder model p(x | z) and the prior p(z). Our work differs from (Mathieu et al, 2019;Chen et al, 2020;Boenninghoff et al, 2020) in that we consider fat-tailed variational approximations q(z | x) rather than priors p(z). Although (Abiri & Ohlsson, 2020) also considers a StudentT approximate posterior, our work involves a more general variational family which use normalizing flows.…”

“…Fat-tails in variational inference. Recent work in variational autoencoders (VAEs) have considered relaxing Gaussian assumptions to heavier-tailed distributions (Mathieu et al, 2019;Chen et al, 2019;Boenninghoff et al, 2020;Abiri & Ohlsson, 2020). In (Mathieu et al, 2019), a Stu-dentT prior distribution p(z) is considered over the latent code z in a VAE with Gaussian encoder q(z | x).…”

“…It showed improvements in the marginal likelihoods of reconstructed images. In addition, Boenninghoff et al (2020) consider a mixture of StudentTs for the prior p(z). To position our work in context, note that the encoder q(z | x) may be viewed as a variational approximation to the posterior p(z | x) defined by the decoder model p(x | z) and the prior p(z).…”