Estimating Image Depth in the Comics Domain

Bhattacharjee, Deblina; Everaert, Martin; Salzmann, Mathieu; Süsstrunk, Sabine

doi:10.1109/wacv51458.2022.00118

Cited by 5 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other instance weighting methods in the shallow regime, [14,74,75] have been explored for a different range of applications. Some specific vision applications have been explored in the context of UDA [31,34,36,50] , such as monocular depth estimation [1,5,43,63] or gaze estimation [4,26]. However, these methods aim at improving upon a specific task and not for regression tasks in general.…”

Section: Related Workmentioning

confidence: 99%

DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices

Nejjar¹,

Wang²,

Fink³

2023

Preprint

View full text Add to dashboard Cite

Unsupervised Domain Adaptation Regression (DAR) aims to bridge the domain gap between a labeled source dataset and an unlabelled target dataset for regression problems. Recent works mostly focus on learning a deep feature encoder by minimizing the discrepancy between source and target features. In this work, we present a different perspective for the DAR problem by analyzing the closed-form ordinary least square (OLS) solution to the linear regressor in the deep domain adaptation context. Rather than aligning the original feature embedding space, we propose to align the inverse Gram matrix of the features, which is motivated by its presence in the OLS solution and the Gram matrix's ability to capture the feature correlations. Specifically, we propose a simple yet effective DAR method which leverages the pseudo-inverse lowrank property to align the scale and angle in a selected subspace generated by the pseudo-inverse Gram matrix of the two domains. We evaluate our method on three domain adaptation regression benchmarks. Experimental results demonstrate that our method achieves state-of-the-art performance. Our code is available at https://github. com/ismailnejjar/DARE-GRAM .

show abstract

Section: Related Workmentioning

confidence: 99%

DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices

Nejjar¹,

Wang²,

Fink³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…With the development of image style transfer and its connection with domain adaptation, recently [3] adopted style transfer and adversarial training to estimate depth on comics. In essence, the style transfer [4] technique helps them to leverage models trained with large amounts of realworld ground-truth data.…”

Section: Multitask Learning With Vision Transformersmentioning

confidence: 99%

“…Following this, we apply our MTL model on the translated images. We evaluate the performance on the DCM validation set that contains dense depth annotations for 300 DCM comics images and was introduced by [3]. For evaluating on semantic segmentation, we use the OpenCV CVAT interface [11], leveraging the semantic labels of MS-Coco.…”

Section: Datasetsmentioning

confidence: 99%

“…Typically, the existing studies that investigate the various elements in comics are restricted to speech balloon segmentation [1,20,33], text detection [32], panel detection [31,34,46], comics character detection [10,26], and region of interest detection [29]. Recently, [3], presented a depth estimation method on comics by exploiting the scene context. While these methods achieve promising results, they do not produce strong cues for reconfiguring the comics.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dense Multitask Learning to Reconfigure Comics

Bhattacharjee

Süsstrunk

Salzmann

2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

Self Cite

View full text Add to dashboard Cite

In this paper, we develop a MultiTask Learning (MTL) model to achieve dense predictions for comics panels to, in turn, facilitate the transfer of comics from one publication channel to another by assisting authors in the task of reconfiguring their narratives. Our MTL method can successfully identify the semantic units as well as the embedded notion of 3D in comics panels. This is a significantly challenging problem because comics comprise disparate artistic styles, illustrations, layouts, and object scales that depend on the author's creative process. Typically, dense image-based prediction techniques require a large corpus of data. Finding an automated solution for dense prediction in the comics domain, therefore, becomes more difficult with the lack of ground-truth dense annotations for the comics images. To address these challenges, we develop the following solutions-we leverage a commonlyused strategy known as unsupervised image-to-image translation, which allows us to utilize a large corpus of realworld annotations; -we utilize the results of the translations to develop our multitasking approach that is based on a vision transformer backbone and a domain transferable attention module; -we study the feasibility of integrating our MTL dense-prediction method with an existing retargeting method, thereby reconfiguring comics.

show abstract