Day-to-night image translation via transfer learning to keep semantic information for driving simulator

Lee, Jinho; Shiotsuka, Daiki; Bang, Geonkyu; Endo, Yuki; Nishimori, Toshiaki; Nakao, Kenta; Kamijo, Shunsuke

doi:10.1016/j.iatssr.2023.04.001

Cited by 5 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lee et al [ 64 ] transfer-learned the weights of semantic segmentation networks to the day-to-night image translation networks.…”

Section: Methodsmentioning

confidence: 99%

“…Figure 4 shows the day-to-night image translation results of each method. CycleGAN [ 22 ], Lee et al’s method [ 64 ], UNIT [ 27 ], and the proposed method generate expressions of streetlights, while other methods do not generate these expressions. Furthermore, our method generates visually more detailed expressions (i.e., color, shape, and position) of the streetlights than CycleGAN [ 22 ], Lee et al’s method [ 64 ] and UNIT [ 27 ].…”

Section: Methodsmentioning

confidence: 99%

“…CycleGAN [ 22 ], Lee et al’s method [ 64 ], UNIT [ 27 ], and the proposed method generate expressions of streetlights, while other methods do not generate these expressions. Furthermore, our method generates visually more detailed expressions (i.e., color, shape, and position) of the streetlights than CycleGAN [ 22 ], Lee et al’s method [ 64 ] and UNIT [ 27 ]. Translation failures in the sky area, sandwiched between buildings, are observed from the results of CycleGAN [ 22 ], AugGAN [ 43 , 44 ], and Lee et al’s method [ 64 ].…”

Section: Methodsmentioning

confidence: 99%

“…Furthermore, our method generates visually more detailed expressions (i.e., color, shape, and position) of the streetlights than CycleGAN [ 22 ], Lee et al’s method [ 64 ] and UNIT [ 27 ]. Translation failures in the sky area, sandwiched between buildings, are observed from the results of CycleGAN [ 22 ], AugGAN [ 43 , 44 ], and Lee et al’s method [ 64 ]. UNIT [ 27 ] and MUNIT [ 30 ] appear to fail in their translations, as they darken the entire scene, giving the impression of inverting the colors in the images.…”

Section: Methodsmentioning

confidence: 99%

“… Qualitative comparison with related works. The figure shows, from left to right, the input images in the source domain, followed by the day-to-night image translation results from CycleGAN [ 22 ], SemGAN [ 41 ], AugGAN [ 43 , 44 ], Lee et al [ 64 ], UNIT [ 27 ], MUNIT [ 30 ], and our method, respectively. …”

Section: Figurementioning

confidence: 99%

See 4 more Smart Citations

Semantic and Geometric-Aware Day-to-Night Image Translation Network

Bang,

Lee,

Endo

et al. 2024

Sensors

Self Cite

View full text Add to dashboard Cite

Autonomous driving systems heavily depend on perception tasks for optimal performance. However, the prevailing datasets are primarily focused on scenarios with clear visibility (i.e., sunny and daytime). This concentration poses challenges in training deep-learning-based perception models for environments with adverse conditions (e.g., rainy and nighttime). In this paper, we propose an unsupervised network designed for the translation of images from day-to-night to solve the ill-posed problem of learning the mapping between domains with unpaired data. The proposed method involves extracting both semantic and geometric information from input images in the form of attention maps. We assume that the multi-task network can extract semantic and geometric information during the estimation of semantic segmentation and depth maps, respectively. The image-to-image translation network integrates the two distinct types of extracted information, employing them as spatial attention maps. We compare our method with related works both qualitatively and quantitatively. The proposed method shows both qualitative and qualitative improvements in visual presentation over related work.

show abstract

“…Lee et al [ 64 ] transfer-learned the weights of semantic segmentation networks to the day-to-night image translation networks.…”

Section: Methodsmentioning

confidence: 99%