The objective is to improve the recognition rate of white and near cotton-coloured impurities in raw cotton against a single white visible light source background. A lightweight detection network model without anchor boxes based on improved YOLO v4-tiny is proposed in this paper based on weighted feature fusion (WFF). The WFF strategy was used to improve the detection accuracy of the improved YOLO v4-tiny algorithm. Meanwhile, to address the disadvantage that the anchor boxes obtained by the K-means algorithm clustering do not have global features, the anchor-free and tiny decoupled head schemes are used instead of the traditional coupled detection head. The improved algorithm was validated on the PASCAL VOC2012 dataset and the self-built raw cotton impurity dataset collected by high-speed camera. On the raw cotton impurity test set, compared to the original YOLO v4-tiny model, the mean average precision (mAP) and frames per second (FPS) of the improved model increased by 10.35% and 6.9%, respectively. The proposed model detects white and near cotton-coloured impurities with an accuracy of up to 98.78% and 98.00%. The experimental results show that the proposed method can effectively detect and identify impurities in raw cotton, which is of great practical significance for foreign matter detection and identification in cotton.