Relatable Clothing: Soft-Attention Mechanism for Detecting Worn/Unworn Objects

Abstract

We have identified a need in visual relationship detection and biometrics related research for a dataset and model which focuses on person-clothing pairs. Previous to our Relatable Clothing dataset, there were no publicly available datasets usable for "worn" and "unworn" clothing detection. In this paper we propose a novel visual relationship model architecture for "worn" and "unworn" clothing detection that makes use of a soft attention mechanism for feature fusion between a conventional ResNet backbone and our novel person-clothing mask feature extraction architecture. The best proposed model achieves 98.62% accuracy, 99.50% precision, 98.31% recall, and 99.14% specificity on the Relatable Clothing dataset, outperforming our previous iterations. We release our models which can be found on the Relatable Clothing GitHub repository for future research and applications into detecting and analyzing person-clothing pairs.

A Survey on Tools and Techniques for Localizing Abnormalities in X-ray Images Using Deep Learning

“…Soft attention is the most popular branch that offers flexibility and ease of implementation [111]. Its applications can be found in many fields of computer vision, e.g., classification [112,113], object detection [114], segmentation [115,116], model generation [111], etc. The mechanism can be further divided into sub-fields.…”