A Method for Target Detection Based on Synthetic Samples of Digital Twins

Abstract

Machine vision technology for target detection is crucial in industrial production and manufacturing. Industries may improve productivity with accurate target detection, which requires abundant samples for training; however, it is expensive and difficult to obtain because of enterprise data privacy and security constraints. This paper proposes a method for quickly creating synthetic samples based on digital twins as a solution to the challenge. First, utilizing the virtual engine to replicate the real detecting environment, generate a range of sample photos, and extract the target object's three-dimensional coordinates in the virtual scene. Subsequently, an annotation method is designed for synthetic samples obtained from the virtual scene, utilizing principles of three-dimensional coordinate transformation and perspective coordinate transformation. This method efficiently produces numerous labeled samples with diverse annotations. Ultimately, the model performs detection tasks in the actual world using the synthetic samples as training data. The experimental results show that the synthetic samples created by this method based on digital twins can substitute real samples and effectively identify target objects during actual detection tasks. This paper proposes a unique synthetic samples strategy that decreases sample collection costs and privacy risks and solves machine vision detection technology's sample limitations-induced detection performance drop.