KitWaSor: Pioneering pre‐trained model for kitchen waste sorting with an innovative million‐level benchmark dataset

Abstract

Intelligent sorting is an important prerequisite for the full quantitative consumption and harmless disposal of kitchen waste. The existing object detection method based on an ImageNet pre‐trained model is an effective way of sorting. Owing to significant domain gaps between natural images and kitchen waste images, it is difficult to reflect the characteristics of diverse scales and dense distribution in kitchen waste based on an ImageNet pre‐trained model, leading to poor generalisation. In this article, the authors propose the first pre‐trained model for kitchen waste sorting called KitWaSor, which combines both contrastive learning (CL) and masked image modelling (MIM) through self‐supervised learning (SSL). First, to address the issue of diverse scales, the authors propose a mixed masking strategy by introducing an incomplete masking branch based on the original random masking branch. It prevents the complete loss of small‐scale objects while avoiding excessive leakage of large‐scale object pixels. Second, to address the issue of dense distribution, the authors introduce semantic consistency constraints on the basis of the mixed masking strategy. That is, object semantic reasoning is performed through semantic consistency constraints to compensate for the lack of contextual information. To train KitWaSor, the authors construct the first million‐level kitchen waste dataset across seasonal and regional distributions, named KWD‐Million. Extensive experiments show that KitWaSor achieves state‐of‐the‐art (SOTA) performance on the two most relevant downstream tasks for kitchen waste sorting (i.e. image classification and object detection), demonstrating the effectiveness of the proposed KitWaSor.