ImageCLEF 2020: Multimedia Retrieval in Lifelogging, Medical, Nature, and Internet Applications

Ionescu, Bogdan; Müller, Henning; Péteri, Renaud; Dang-Nguyen, Duc-Tien; Zhou, Liting; Piras, Luca; Riegler, Michael; Halvorsen, Pål; Tran, Minh-Triet; Lux, Mathias; Gurrin, Cathal; Chamberlain, Jon; Clark, Adrian; Campello, Antonio; García, Narciso; Abacha, Asma Ben; Datla, Vivek V.; Hasan, Sadid A.; Liu, Joey; Demner‐Fushman, Dina; Pelka, Obioma; Friedrich, Christoph M.; Cid, Yashin Dicente; Kozlovski, Serge; Liauchuk, Vitali; Kovalev, Vassili; Berari, Raul; Brie, Paul; Fichou, Dimitri; Dogariu, Mihai; Stefan, Liviu Daniel; Constantin, Mihai Gabriel

doi:10.1007/978-3-030-45442-5_69

Cited by 2 publications

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…use of engineered features like Scale-Invariant Feature Transform (SIFT) to Convolutional Neural Networks (CNNs) [33,56]. It has been applied to many different specialized domains like artworks and cultural heritage [4,12], commerce [17,54], surveillance [2], nature [24,25]. In the basic form, the query is composed of only an image, of which features are computed and compared with the ones extracted by a database of images.…”

Section: :2mentioning

confidence: 99%

Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features

Baldrati,

Bertini,

Uricchio

et al. 2023

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

Given a query composed of a reference image and a relative caption, the Composed Image Retrieval goal is to retrieve images visually similar to the reference one that integrates the modifications expressed by the caption. Given that recent research has demonstrated the efficacy of large-scale vision and language pre-trained (VLP) models in various tasks, we rely on features from the OpenAI CLIP model to tackle the considered task. We initially perform a task-oriented fine-tuning of both CLIP encoders using the element-wise sum of visual and textual features. Then, in the second stage, we train a Combiner network that learns to combine the image-text features integrating the bimodal information and providing combined features used to perform the retrieval. We use contrastive learning in both stages of training. Starting from the bare CLIP features as a baseline, experimental results show that the task-oriented fine-tuning and the carefully crafted Combiner network are highly effective and outperform more complex state-of-the-art approaches on FashionIQ and CIRR, two popular and challenging datasets for composed image retrieval. Code and pre-trained models are available at https://github.com/ABaldrati/CLIP4Cir .

show abstract

Section: :2mentioning

confidence: 99%

Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features

Baldrati,

Bertini,

Uricchio

et al. 2023

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

show abstract

Report on CLEF 2020

et al. 2020

View full text Add to dashboard Cite

This is a report on the tenth edition of the Conference and Labs of the Evaluation Forum (CLEF 2020), (virtually) held from September 22--25, 2020, in Thessaloniki, Greece. CLEF was a four day event combining a Conference and an Evaluation Forum. The Conference featured keynotes by Ellen Voorhees and Yiannis Kompasiaris, and presentation of peer reviewed research papers covering a wide range of topics in addition to many posters. The Evaluation Forum consisted to twelve Labs: ARQMath, BioASQ, CheckThat!, ChEMU, CLEF eHealth, eRisk, HIPE, ImageCLEF, LifeCLEF, LiLAS, PAN, and Touché, addressing a wide range of tasks, media, languages, and ways to go beyond standard test collections.

show abstract

ImageCLEF 2020: Multimedia Retrieval in Lifelogging, Medical, Nature, and Internet Applications

Cited by 2 publications

References 18 publications

Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features

Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features

Report on CLEF 2020

Contact Info

Product

Resources

About