Unsupervised Learning of Disentangled Representation via Auto-Encoding: A Survey

Abstract

In recent years, the rapid development of deep learning approaches has paved the way to explore the underlying factors that explain the data. In particular, several methods have been proposed to learn to identify and disentangle these underlying explanatory factors in order to improve the learning process and model generalization. However, extracting this representation with little or no supervision remains a key challenge in machine learning. In this paper, we provide a theoretical outlook on recent advances in the field of unsupervised representation learning with a focus on auto-encoding-based approaches and on the most well-known supervised disentanglement metrics. We cover the current state-of-the-art methods for learning disentangled representation in an unsupervised manner while pointing out the connection between each method and its added value on disentanglement. Further, we discuss how to quantify disentanglement and present an in-depth analysis of associated metrics. We conclude by carrying out a comparative evaluation of these metrics according to three criteria, (i) modularity, (ii) compactness and (iii) informativeness. Finally, we show that only the Mutual Information Gap score (MIG) meets all three criteria.

Compression of molecular fingerprints with autoencoder networks

“…They are also suited for nonlinear dimensionality reduction, e. g ., for data visualization [8] or data embedding for downstream tasks [9, 10]. Autoencoders enable unsupervised representation learning [11]. For example, they have been used to generate data‐driven molecular vector representations based on SMILES strings [12].…”

Learning disentangled representation of video for pallet decomposition in industrial warehouses

“…Disentangled representation learning: Disentangled representation learning 1,2,17,18 refers to the process of extracting a low-dimensional representation of the data that captures the most abstract and independent factors of variation. The intention behind this representation is to improve performance on a variety of downstream tasks such as video prediction.…”

A Data Compression Method for Wellbore Stability Monitoring Based on Deep Autoencoder

“…Comprising two main components—the encoder and the decoder—the autoencoder operates by first reducing the dimensionality of the data, thereby reducing the volume of data and facilitating data compression. Subsequently, the decoder reconstructs the data to closely resemble the input data [ 30 , 31 ]. Leveraging its capacity for dimensionality reduction and reconstruction, autoencoders find widespread application in data compression.…”