Scene Graph Expansion for Semantics-Guided Image Outpainting

Yang, Chiao-An; Tan, Cheng-Yo; Fan, Wan-Cyuan; Yang, Cheng–Fu; Wu, Meng-Lin; Wang, Yu-Chiang Frank

doi:10.1109/cvpr52688.2022.01517

Cited by 14 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Region-based Relations Mining. Region-based relations mining is popular in visual tasks, which is widely used in video classification (Wang and Gupta 2018), segmentation , tracking (Gao, Zhang, and Xu 2019) and image outpainting (Yang et al 2022), etc. In the image cropping area, yet, rare works attempt to model the visual regional correlations.…”

Section: Related Workmentioning

confidence: 99%

Spatial-Semantic Collaborative Cropping for User Generated Content

Su,

Cao,

Deng

et al. 2024

AAAI

View full text Add to dashboard Cite

A large amount of User Generated Content (UGC) is uploaded to the Internet daily and displayed to people world-widely through the client side (mobile and PC). This requires the cropping algorithms to produce the aesthetic thumbnail within a specific aspect ratio on different devices. However, existing image cropping works mainly focus on landmark or landscape images, which fail to model the relations among the multi-objects with the complex background in UGC. Besides, previous methods merely consider the aesthetics of the cropped images while ignoring the content integrity, which is crucial for UGC cropping. In this paper, we propose a Spatial-Semantic Collaborative cropping network (S2CNet) for arbitrary user generated content accompanied by a new cropping benchmark. Specifically, we first mine the visual genes of the potential objects. Then, the suggested adaptive attention graph recasts this task as a procedure of information association over visual nodes. The underlying spatial and semantic relations are ultimately centralized to the crop candidate through differentiable message passing, which helps our network efficiently to preserve both the aesthetics and the content integrity. Extensive experiments on the proposed UGCrop5K and other public datasets demonstrate the superiority of our approach over state-of-the-art counterparts.

show abstract

Section: Related Workmentioning

confidence: 99%

Spatial-Semantic Collaborative Cropping for User Generated Content

Su,

Cao,

Deng

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Cross-modal image generation has seen a lot of attention in recent years with work on text-to-image models [12,[32][33][34]40], speech-toimage models [5,20], and image-to-video models [42]. Others have focused on using more direct representations to control generation, including generation from semantic masks [18,26,52], or worked to convert higher-level representations such as text [15,30], scene graphs [7,19,44,51], and bounding boxes [22] into such a semantic mask. There have also been several attempts at learning spatially disentangled scene representations by composing latent features that correspond to specific parts of a scene [10,27].…”

Section: Related Workmentioning

confidence: 99%

Street-View Image Generation from a Bird's-Eye View Layout

Swerdlow¹,

Xu²,

Zhou³

2023

Preprint

View full text Add to dashboard Cite

Bird's-Eye View (BEV) Perception has received increasing attention in recent years as it provides a concise and unified spatial representation across views and benefits a diverse set of downstream driving applications. While the focus has been placed on discriminative tasks such as BEV segmentation, the dual generative task of creating streetview images from a BEV layout has rarely been explored. The ability to generate realistic street-view images that align with a given HD map and traffic layout is critical for visualizing complex traffic scenarios and developing robust perception models for autonomous driving. In this paper, we propose BEVGen, a conditional generative model that synthesizes a set of realistic and spatially consistent surrounding images that match the BEV layout of a traffic scenario. BEVGen incorporates a novel cross-view transformation and spatial attention design which learn the relationship between cameras and map views to ensure their consistency. Our model can accurately render road and lane lines, as well as generate traffic scenes under different weather conditions and times of day. The code will be made publicly available.

show abstract