On The State of Data In Computer Vision: Human Annotations Remain Indispensable for Developing Deep Learning Models

Abstract

High-quality labeled datasets play a crucial role in fueling the development of machine learning (ML), and in particular the development of deep learning (DL). However, since the emergence of the ImageNet dataset and the AlexNet model in 2012, the size of new open-source labeled vision datasets has remained roughly constant. Consequently, only a minority of publications in the computer vision community tackle supervised learning on datasets that are orders of magnitude larger than Imagenet. In this paper, we survey computer vision research domains that study the effects of such large datasets on model performance across different vision tasks. We summarize the community's current understanding of those effects, and highlight some open questions related to training with massive datasets. In particular, we tackle: (a) The largest datasets currently used in computer vision research and the interesting takeaways from training on such datasets; (b) The effectiveness of pretraining on large datasets; (c) Recent advancements and hurdles facing synthetic datasets; (d) An overview of double descent and sample nonmonotonicity phenomena; and finally, (e) A brief discussion of lifelong/continual learning and how it fares compared to learning from huge labeled datasets in an offline setting. Overall, our findings are that research on optimization for deep learning focuses on perfecting the training routine and thus making DL models less data hungry, while research on synthetic datasets aims to offset the cost of data labeling. However, for the time being, acquiring non-synthetic labeled data remains indispensable to boost performance.

PCCAU-Net: A Novel Road Extraction Method Based on Coord Convolution and a DCA Module

“…In the diverse landscape of deep learning and computer vision, the fidelity and precision of model performance are predominantly dictated by the quality of datasets and meticulously engineered preprocessing pipelines [41]. To rigorously assess the efficacy of our proposed model-especially its adaptability across multiple scales and contexts-we elected to utilize two industry-standard road extraction datasets: the Massachusetts Roads Dataset [42] and the DeepGlobe Road Dataset [43].…”