Synthetic Examples Improve Generalization for Rare Classes

Abstract

The ability to detect and classify rare occurrences in images has important applications-for example, counting rare and endangered species when studying biodiversity, or detecting infrequent traffic scenarios that pose a danger to self-driving cars. Few-shot learning is an open problem: current computer vision systems struggle to categorize objects they have seen only rarely during training, and collecting a sufficient number of training examples of rare events is often challenging and expensive, and sometimes outright impossible. We explore in depth an approach to this problem: complementing the few available training images with ad-hoc simulated data. Our testbed is animal species classification, which has a real-world long-tailed distribution. We analyze the effect of different axes of variation in simulation, such as pose, lighting, model, and simulation method, and we prescribe best practices for efficiently incorporating simulated data for real-world performance gain. Our experiments reveal that synthetic data can considerably reduce error rates for classes that are rare, that as the amount of simulated data is increased, accuracy on the target class improves, and that high variation of simulated data provides maximum performance gain.

Recognizing New Classes with Synthetic Data in the Loop: Application to Traffic Sign Recognition

“…Finally, the work with the most similar goal to that of this paper has recently been presented by Beery et al [99]. The addressed application is animal detection and classification from static cameras.…”

Synthesize & Learn: Jointly Optimizing Generative and Classifier Networks for Improved Drowsiness Detection

“…Surprisingly, the joint training also improves C's performance on the Alert class, suggesting well separated boundaries between all three classes are formed when synthetic samples are added during training. The synthetic samples can stretch the feature manifold [21] and push real samples from different classes further away from each other, as visualized in Figure 2, improving inter-class separation and consequently model performance.…”

“…Fig.2: tSNE visualization: As can be seen, the drowsiness classes are deeply entangled (left) but synthetic vectors stretch the manifold pushing real samples further away from each other (right), similar to[21]. Consequently, jointly training the classifier network with the GAN improves its performance.…”

Scaling Biodiversity Monitoring for the Data Age

“…New methodologies are being developed to tackle these challenges, with expert ecologists providing data, guidance, and much-needed context for different sensor types and taxa. Exciting recent work includes using synthesized data for rare species to improve rareclass performance [7], incorporating learned geospatial priors to improve camera trap data due to a combination of community building, data science, and machine learning research. The MegaDetector is trained to localize animals but not predict their species, which has been shown to be more robust to both new species and new camera deployments than species-specific models [6].…”