Abstract:Typical semantic segmentation methods do not recognize unknown pixels during the test or deployment stage. This capability is critical for open-world environment applications where unseen objects appear all the time. Recently, to solve those limitations, Open Set Semantic Segmentation (OSSS) was introduced. This task aims to produce known and unknown pixels semantic segments. However, due to its recent introduction, few works are found in the literature, and consequently, few datasets are publicly available. T… Show more
“…However, these approaches are still vulnerable to feature collapse [ 22 ]. We direct the reader to [ 64 , 65 ] for a broader overview of open-set recognition. Open-world approaches attempt to disentangle the detected unknown concepts towards new semantic classes.…”
Standard machine learning is unable to accommodate inputs which do not belong to the training distribution. The resulting models often give rise to confident incorrect predictions which may lead to devastating consequences. This problem is especially demanding in the context of dense prediction since input images may be only partially anomalous. Previous work has addressed dense out-of-distribution detection by discriminative training with respect to off-the-shelf negative datasets. However, real negative data may lead to over-optimistic evaluation due to possible overlap with test anomalies. To this end, we extend this approach by generating synthetic negative patches along the border of the inlier manifold. We leverage a jointly trained normalizing flow due to a coverage-oriented learning objective and the capability to generate samples at different resolutions. We detect anomalies according to a principled information-theoretic criterion which can be consistently applied through training and inference. The resulting models set the new state of the art on benchmarks for out-of-distribution detection in road-driving scenes and remote sensing imagery despite minimal computational overhead.
“…However, these approaches are still vulnerable to feature collapse [ 22 ]. We direct the reader to [ 64 , 65 ] for a broader overview of open-set recognition. Open-world approaches attempt to disentangle the detected unknown concepts towards new semantic classes.…”
Standard machine learning is unable to accommodate inputs which do not belong to the training distribution. The resulting models often give rise to confident incorrect predictions which may lead to devastating consequences. This problem is especially demanding in the context of dense prediction since input images may be only partially anomalous. Previous work has addressed dense out-of-distribution detection by discriminative training with respect to off-the-shelf negative datasets. However, real negative data may lead to over-optimistic evaluation due to possible overlap with test anomalies. To this end, we extend this approach by generating synthetic negative patches along the border of the inlier manifold. We leverage a jointly trained normalizing flow due to a coverage-oriented learning objective and the capability to generate samples at different resolutions. We detect anomalies according to a principled information-theoretic criterion which can be consistently applied through training and inference. The resulting models set the new state of the art on benchmarks for out-of-distribution detection in road-driving scenes and remote sensing imagery despite minimal computational overhead.
“…OSS is an inherently harder problem due to its dense labeling nature compared to Openset Recognition or Classification. Thus, in real-world scenarios, it is harder to perform open-set semantic segmentation precisely (Brilhador et al, 2021). The complexity of the problem may explain why there is still a gap in the literature, with only a handful of articles tackling the issue (Cui et al, 2020).…”
Monteiro Nunes, Ian; Soledade Poggi de Aragão, Marcus Vinicius (Advisor); Neves de Oliveira, Hugo (Co-Advisor). Open-set semantic segmentation for remote sensing images. Rio de Janeiro, 2023. 211p.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.