Unconstrained Foreground Object Search

Zhao, Yinan; Price, Brian; Cohen, Scott; Gurari, Danna

doi:10.1109/iccv.2019.00212

Cited by 11 publications

(39 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This work is the closest to ours and serves as the baseline method for comparison purpose. More recently, Zhao et al [26] proposed an unconstrained FoS task that aims to retrieve universal compatible foreground without specifying its category. We only focus on the constrained FoS problem with known foreground category in this work.…”

Section: Foreground Object Searchmentioning

confidence: 99%

“…For object insertion in photo editing, users often find it challenging and time-consuming to acquire compatible foregrounds in a foreground pool. Object insertion can be used to fill a new foreground to a region comprising undesired objects in the background [26].…”

Section: Introductionmentioning

confidence: 99%

“…The second problem is that given a query input and a foreground instance, how to define and decide their compatibility. Most recent methods [25] [26] jointly learned foreground similarity and query-foreground compatibility without decoupling the two problems. It makes the results difficult to interpret.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Interpretable Foreground Object Search As Knowledge Distillation

Zhuang

et al. 2020

Preprint

View full text Add to dashboard Cite

This paper proposes a knowledge distillation method for foreground object search (FoS). Given a background and a rectangle specifying the foreground location and scale, FoS retrieves compatible foregrounds in a certain category for later image composition. Foregrounds within the same category can be grouped into a small number of patterns. Instances within each pattern are compatible with any query input interchangeably. These instances are referred to as interchangeable foregrounds. We first present a pipeline to build pattern-level FoS dataset containing labels of interchangeable foregrounds. We then establish a benchmark dataset for further training and testing following the pipeline. As for the proposed method, we first train a foreground encoder to learn representations of interchangeable foregrounds. We then train a query encoder to learn query-foreground compatibility following a knowledge distillation framework. It aims to transfer knowledge from interchangeable foregrounds to supervise representation learning of compatibility. The query feature representation is projected to the same latent space as interchangeable foregrounds, enabling very efficient and interpretable instance-level search. Furthermore, pattern-level search is feasible to retrieve more controllable, reasonable and diverse foregrounds. The proposed method outperforms the previous state-of-theart by 10.42% in absolute difference and 24.06% in relative improvement evaluated by mean average precision (mAP). Extensive experimental results also demonstrate its efficacy from various aspects. The benchmark dataset and code will be release shortly.

show abstract

Section: Foreground Object Searchmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Interpretable Foreground Object Search As Knowledge Distillation

Zhuang

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Specifying the category sets a limit on the search space, thus preventing the system from recommending diverse sets of objects. Later, UFO [28] proposes an unconstrained search method, i.e. objects from all categories are considered as candidates for retrieval.…”

Section: Introductionmentioning

confidence: 99%

“…The unconstrained setting is closer to real-world scenarios, which require a large and diverse foreground object gallery to satisfy different users. However, UFO [28] focuses on finding semantically compatible foreground object and does not explicitly model lighting and geometry, which are critical factors for making object compositing realistic, as shown in Figs. 1 and 2.…”

Section: Introductionmentioning

confidence: 99%

GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing

Zhu¹,

Lin²,

Cohen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Compositing-aware object search aims to find the most compatible objects for compositing given a background image and a query bounding box. Previous works focus on learning compatibility between the foreground object and background, but fail to learn other important factors from large-scale data, i.e. geometry and lighting. To move a step further, this paper proposes GALA (Geometry-and-Lighting-Aware), a generic foreground object search method with discriminative modeling on geometry and lighting compatibility for open-world image compositing. Remarkably, it achieves state-of-the-art results on the CAIS dataset and generalizes well on large-scale open-world datasets, i.e. Pixabay and Open Images. In addition, our method can effectively handle non-box scenarios, where users only provide background images without any input bounding box. A web demo (see supplementary materials) is built to showcase applications of the proposed method for compositing-aware search and automatic location/scale prediction for the foreground object.

show abstract

Interpretable Foreground Object Search as Knowledge Distillation

Zhuang

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Unconstrained Foreground Object Search

Cited by 11 publications

References 42 publications

Interpretable Foreground Object Search As Knowledge Distillation

Interpretable Foreground Object Search As Knowledge Distillation

GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing

Interpretable Foreground Object Search as Knowledge Distillation

Contact Info

Product

Resources

About