StyleBabel: Artistic Style Tagging and Captioning

Ruta, Dan; Gilbert, A.; Aggarwal, Pranav; Marri, Naveen; Kale, Ajinkya; Briggs, Jo; Speed, Chris; Jin, Hailin; Faieta, Baldo; Filipkowski, Alex; Lin, Zhe; Collomosse, John

doi:10.48550/arxiv.2203.05321

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Knowledge-based deep learning models are broadly divided into two aspects: (1) external knowledge introduction [15], (2) internal knowledge mining [26]. The former leverages expert knowledge by introducing external data [4,35,49] or pre-trained models [12,64,83,91]. The latter means constructing correlations of training data by similarity [17,21,55] or discovering favorable substructures of internal models [7,38,39,84].…”

Section: Knowledge-based Modelsmentioning

confidence: 99%

UKnow: A Unified Knowledge Protocol for Common-Sense Reasoning and Vision-Language Pre-training

Gong¹,

Xie²,

Feng³

et al. 2023

Preprint

View full text Add to dashboard Cite

This work presents a unified knowledge protocol, called UKnow, which facilitates knowledge-based studies from the perspective of data. Particularly focusing on visual and linguistic modalities, we categorize data knowledge into five unit types, namely, in-image, intext, cross-image, cross-text, and image-text. Following this protocol, we collect, from public international news, a large-scale multimodal knowledge graph dataset that consists of 1,388,568 nodes (with 571,791 visionrelated ones) and 3,673,817 triplets. The dataset is also annotated with rich event tags, including 96 coarse labels and 9,185 fine labels, expanding its potential usage. To further verify that UKnow can serve as a standard protocol, we set up an efficient pipeline to help reorganize existing datasets under UKnow format. Finally, we benchmark the performance of some widely-used baselines on the tasks of common-sense reasoning and visionlanguage pre-training. Results on both our new dataset and the reformatted public datasets demonstrate the effectiveness of UKnow in knowledge organization and method evaluation. Code, dataset, conversion tool, and baseline models will be made public. (https://github. com/Gongggg/UKnow)

show abstract