Parameter-efficient Model Adaptation for Vision Transformers

He, Xiahong; Li, Chunyuan; Zhang, Pengchuan; Yang, Jun; Wang, Xin Eric

doi:10.48550/arxiv.2203.16329

Cited by 7 publications

(7 citation statements)

References 32 publications

(70 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their WRN-based approach achieved a remarkable accuracy of 99.17%, setting a new benchmark in efficiency and accuracy. Recently, other variants, including hierarchical ViTs with diverse resolutions and spatial embeddings [44], have been proposed. Without a doubt, the advancements in large ViTs underscore the importance of developing efficient model adaptation strategies.…”

Section: Related Workmentioning

confidence: 99%

Mapping of Land Use and Land Cover (LULC) Using EuroSAT and Transfer Learning

Kunwar,

Ferdush

2024

RIG

View full text Add to dashboard Cite

As the global population continues to expand, the demand for natural resources increases. Unfortunately, human activities account for 23% of greenhouse gas emissions. On a positive note, remote sensing technologies have emerged as a valuable tool in managing our environment. These technologies allow us to monitor land use, plan urban areas, and drive advancements in areas such as agriculture, climate change mitigation, disaster recovery, and environmental monitoring. Recent advances in Artificial Intelligence (AI), computer vision, and earth observation data have enabled unprecedented accuracy in land use mapping. By using transfer learning and fine-tuning with red-green-blue (RGB) bands, we achieved an impressive 99.19% accuracy in land use analysis. Such findings can be used to inform conservation and urban planning policies.

show abstract

Section: Related Workmentioning

confidence: 99%

Mapping of Land Use and Land Cover (LULC) Using EuroSAT and Transfer Learning

Kunwar,

Ferdush

2024

RIG

View full text Add to dashboard Cite

show abstract

“…For VLP models, it provides a unique opportunity to leverage the text encoders for model adaptation, including Conditional Prompt Learning (Zhou et al, 2022b), Color Prompt Tuning (CPT) (Yao et al, 2021), VL-Adapter (Sung et al, 2022b), and CLIP Adapter (Gao et al, 2021). A comprehensive study on parameter efficiency can be found in He et al (2022). • Robustness.…”

Section: Advanced Topicsmentioning

confidence: 99%

Vision-Language Pre-training: Basics, Recent Advances, and Future Trends

Gan¹,

Fu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper surveys vision-language pre-training (VLP) methods for multimodal intelligence that have been developed in the last few years. We group these approaches into three categories: (i) VLP for image-text tasks, such as image captioning, image-text retrieval, visual question answering, and visual grounding; (ii) VLP for core computer vision tasks, such as (open-set) image classification, object detection, and segmentation; and (iii) VLP for video-text tasks, such as video captioning, video-text retrieval, and video question answering. For each category, we present a comprehensive review of state-of-the-art methods, and discuss the progress that has been made and challenges still being faced, using specific systems and models as case studies. In addition, for each category, we discuss advanced topics being actively explored in the research community, such as big foundation models, unified modeling, in-context few-shot learning, knowledge, robustness, and computer vision in the wild, to name a few.♠ Zhe Gan and Jianfeng Gao initiated the project. Zhe Gan and Linjie Li took lead in the writing of Chapter 1. Linjie Li and Jianfeng Gao took lead in the writing of Chapter 2. Zhe Gan further took lead in the writing of Chapter 3 and 7. Chunyuan Li took lead in the writing of Chapter 4. Linjie Li further took lead in the writing of Chapter 5. Lijuan Wang and Zicheng Liu took lead in the writing of Chapter 6. All the authors provided project advice, and contributed to paper editing and proofreading.

show abstract

“…Very recently, the Segmentation Anything Model (SAM) [21] has gained massive attention as a powerful and general vision segmentation model capable of generating various and fine-grained segmentation masks conditioned by the user prompt. Despite its strong performance over natural images, many recent studies studies have shown that Adaption can be easily adopted in various downstream computer vision tasks [4,17]. Therefore, we believe Adaption is the most fitting technique for carrying SAM to the medical domain.…”

Section: Introductionmentioning

confidence: 99%