Deep CNN based online image deduplication technique for cloud storage system

Kaur, Ravneet; Bhattacharya, Jhilik; Chana, Inderveer

doi:10.1007/s11042-022-13182-7

Cited by 7 publications

(3 citation statements)

References 83 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep featurebased methods leverage Convolutional Neural Networks (CNNs) for image deduplication. For instance, Kaur et al [2] proposed a CNN-based online image deduplication technique, showing superior performance in detecting exact and nearexact images. Kordopatis-Zilos et al [19] introduced a method for near-duplicate video retrieval using unsupervised and supervised approaches based on Deep Metric Learning (DML).…”

Section: Related Workmentioning

confidence: 99%

“…DNNs have demonstrated near-human performance across various vision tasks with tens of millions of images available in the training phase, e.g., ImageNet [1] benchmark. Going beyond known curated image benchmarks, the quality and the percentage of redundancy in a typical dataset are unknown [2]. The SVHN [3] corpus has cropped digits from the house number plates captured by the street view cameras.…”

Section: Introductionmentioning

confidence: 99%

“…It is reasonable to expect repetitive images featuring similar plate materials and identical printed digits. These redundant images are often referred to as duplicates or near-duplicates [2]. Definition 1: Duplicate or near-duplicate images.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Image Deduplication Using Efficient Visual Indexing and Retrieval: Optimizing Storage, Time and Energy for Deep Neural Network Training

Rahman,

Biswas,

Chen

et al. 2024

Preprint

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) have gained widespread popularity for tasks related to visual processing due to their superior performance and the wealth of images and videos available. Rich concept representation in the training dataset is crucial for the effectiveness of the trained deep neural network model. Often, images of the same object taken from slightly different angles or with minor variations are present, and this redundancy is wasteful as the bandwidth, storage, and processing power are limited. Near duplicate images contribute very little to the effectiveness of the model, and here we propose a novel framework for Visual Indexing and Retrieval-based image Deduplication (VIRD). VIRD effectively eliminates redundant data and maintains information quality in the training corpus through visual indexing and retrieval. VIRD balances the tradeoff between a large deduplication ratio and a stable mAP by adjusting the deduplication threshold for graph-based approximate retrieval of near-duplicate images from given target corpora. The effectiveness of VIRD is validated through extensive experiments on well-known Convolutional Neural Network (CNN) benchmarks. While preserving the same validation mean Average Precision (mAP), VIRD can reduce the corpus size by 25.13%. Moreover , by streamlining the training process, VIRD can lower the energy consumption of DNN training by 27.17%, leading to more practical and sustainable DNN training practices.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%