EST-GAN: Enhancing Style Transfer GANs with Intermediate Game Render Passes

Mittermueller, Martina; Ye, Zhanxiang; Hlavacs, Helmut

doi:10.1109/cog51982.2022.9893673

Cited by 3 publications

(9 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several works aim to address the gap between synthetic and real images by improving the photorealism of generated outputs from rendered images. This can be achieved by employing image-to-image translation methods, such as those proposed in [ 13 , 14 ]. Image translation is the ability to learn a mapping function between an input image and a target image in order to transfer it to another domain.…”

Section: Related Workmentioning

confidence: 99%

“…This approach has led to more faithful results when enhancing images from the GTA V dataset. Some works [ 14 ] took inspiration from the Cycle-GAN [ 29 ] method to enhance the realism or change the style of images into an intricately designed video game. Some other works are very promising, such as DUNIT [ 30 ], MGUIT [ 31 ], and InstaFormer [ 32 ].…”

Section: Related Workmentioning

confidence: 99%

“…They assess sensor performance by comparing real and simulated data, such as point clouds and bounding boxes, to measure the disparity between simulation and reality, validating sensor models. Another group enhances image realism using translation methods, often employing GAN-based architectures [ 13 , 14 ]. Our research introduces novel elements, directly analyzing simulation data features to derive fidelity scores, making it applicable to diverse tasks.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Comprehensive Exploration of Fidelity Quantification in Computer-Generated Images

Duminil,

Ieng,

Gruyer

2024

Sensors

View full text Add to dashboard Cite

Generating realistic road scenes is crucial for advanced driving systems, particularly for training deep learning methods and validation. Numerous efforts aim to create larger and more realistic synthetic datasets using graphics engines or synthetic-to-real domain adaptation algorithms. In the realm of computer-generated images (CGIs), assessing fidelity is challenging and involves both objective and subjective aspects. Our study adopts a comprehensive conceptual framework to quantify the fidelity of RGB images, unlike existing methods that are predominantly application-specific. This is probably due to the data complexity and huge range of possible situations and conditions encountered. In this paper, a set of distinct metrics assessing the level of fidelity of virtual RGB images is proposed. For quantifying image fidelity, we analyze both local and global perspectives of texture and the high-frequency information in images. Our focus is on the statistical characteristics of realistic and synthetic road datasets, using over 28,000 images from at least 10 datasets. Through a thorough examination, we aim to reveal insights into texture patterns and high-frequency components contributing to the objective perception of data realism in road scenes. This study, exploring image fidelity in both virtual and real conditions, takes the perspective of an embedded camera rather than the human eye. The results of this work, including a pioneering set of objective scores applied to real, virtual, and improved virtual data, offer crucial insights and are an asset for the scientific community in quantifying fidelity levels.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Comprehensive Exploration of Fidelity Quantification in Computer-Generated Images

Duminil,

Ieng,

Gruyer

2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Style transfer approaches tailored for computer game applications primarily focus on bridging the gap between synthetic and realistic worlds [10], [11]. Richter et al [10] leveraged real-world datasets to improve the photorealism of the Grand Theft Auto V game.…”

Section: Style Transfer For Computer Gamesmentioning

confidence: 99%

“…Whilst 3D computer games encapsulate more difficulties, such as the synthetic nature of the data, they offer additional three-dimensional information that can be exploited [10]- [12]. Previous work has shown that stylizations of game scenes can be significantly improved when employing G-buffer data [10], [11]. Nevertheless, temporal consistency is not addressed.…”

Section: Introductionmentioning

confidence: 99%