Application of Variational AutoEncoder (VAE) Model and Image Processing Approaches in Game Design

Abstract

In recent decades, the Variational AutoEncoder (VAE) model has shown good potential and capability in image generation and dimensionality reduction. The combination of VAE and various machine learning frameworks has also worked effectively in different daily life applications, however its possible use and effectiveness in modern game design has seldom been explored nor assessed. The use of its feature extractor for data clustering has also been minimally discussed in the literature neither. This study first attempts to explore different mathematical properties of the VAE model, in particular, the theoretical framework of the encoding and decoding processes, the possible achievable lower bound and loss functions of different applications; then applies the established VAE model to generate new game levels based on two well-known game settings; and to validate the effectiveness of its data clustering mechanism with the aid of the Modified National Institute of Standards and Technology (MNIST) database. Respective statistical metrics and assessments are also utilized to evaluate the performance of the proposed VAE model in aforementioned case studies. Based on the statistical and graphical results, several potential deficiencies, for example, difficulties in handling high-dimensional and vast datasets, as well as insufficient clarity of outputs are discussed; then measures of future enhancement, such as tokenization and the combination of VAE and GAN models, are also outlined. Hopefully, this can ultimately maximize the strengths and advantages of VAE for future game design tasks and relevant industrial missions.

Variational Autoencoders for Data Augmentation in Clinical Studies

“…The authors developed a generalized extreme value model that served to connect the value-added evaluation approach with choice modeling [28]. In another recent study, the authors applied a VAE model along with image processing methods in game design [29]. This study is considered to be the first to investigate various mathematical properties associated with VAE models.…”

“…This study is considered to be the first to investigate various mathematical properties associated with VAE models. The VAE model demonstrated its proficiency in data clustering, and it was observed to be particularly efficient in generating images that exhibit a certain graphical structure or in managing and creating images that have low resolution demands [29]. Finally, another study presented a novel approach that combined a disentangled VAE with a bidirectional long short-term memory network backend in order to detect anomalies in heart rate data collected during sleep using a wearable device [30].…”

Measure of Similarity between GMMs Based on Autoencoder-Generated Gaussian Component Representations

“…Moreover, empirical results demonstrate that Adam works well in practice and still achieves a competitive performance when compared with newer methods, as can be seen in [43]. The procedure is performed by using the objective defined in (15) over 30 epochs, with a learning rate of 0.001, a learning decay rate decay of 1e − 05, and the feature map regularization term R F MR (F , G) defined in (21), with a regularization penalty value of λ FMR = 10. The first regularization term R(F , G) in ( 15) refers to network weights and is of the same type as in the case of previously described GMM-Autoenc i,AE .…”

“…In other words, estimates in (20) correspond to the autoencoder feature map observations in R r , while the input Gaussian components are in R d . Since it is assumed that there are M such (d + 1) × (d + 1) SPD matrices corresponding to the individual components of all GMMs present in the training phase of some particular ML task, we estimate the expectation in (21) as:…”

“…In [1], the unscented transform is delivered and applied within a speaker recognition task. Besides these, we also highlight the applications of the KL divergence in the specific tasks of game design, where it can be utilized to establish variational autoencoder (VAE) models that generate new game levels [21], or to compare and analyze the game-levels [22], e.g., as the objective function of an evolutionary algorithm to evolve new levels.…”