Boosting Real-Time Driving Scene Parsing With Shared Semantics

Abstract

Real-time scene parsing is a fundamental feature for autonomous driving vehicles with multiple cameras. In this letter we demonstrate that sharing semantics between cameras with different perspectives and overlapped views can boost the parsing performance when compared with traditional methods, which individually process the frames from each camera. Our framework is based on a deep neural network for semantic segmentation but with two kinds of additional modules for sharing and fusing semantics. On the one hand, a semantics sharing module is designed to establish the pixel-wise mapping between the input images. Features as well as semantics are shared by the map to reduce duplicated workload which leads to more efficient computation. On the other hand, feature fusion modules are designed to combine different modal of semantic features, which leverage the information from both inputs for better accuracy. To evaluate the effectiveness of the proposed framework, we have applied our network to a dual-camera vision system for driving scene parsing. Experimental results show that our network outperforms the baseline method on the parsing accuracy with comparable computations.

Thermal infrared image colorization on nighttime driving scenes with spatially-correlated learning

“…The scene perception system provides a rapid decision response to the control system by analyzing the captured images. With the benefits of high-performance visible light imaging techniques, some studies [29][30][31][32][33] have investigated the driving scene perception based on the visible spectrum. For instance, Xiang et al 29 parsed driving scenes by sharing semantics between two cameras with different perspectives and overlapping views.…”

“…With the benefits of high-performance visible light imaging techniques, some studies [29][30][31][32][33] have investigated the driving scene perception based on the visible spectrum. For instance, Xiang et al 29 parsed driving scenes by sharing semantics between two cameras with different perspectives and overlapping views. SFNet-N 30 developed low-light image enhancement and semantic segmentation networks to deal with the boundary blur problem of nighttime images.…”

Scene Adaptation for Semantic Segmentation using Adversarial Learning

“…The algorithm consists of two modules: a segmentation network and a discriminator operating at different levels of the network. Some works address the domain gap created by the device used to capture the data Yang et al (2019) for instance addressed the domain gap between pinhole and omnidirectional perspectives, whereas Xiang et al (2020) addressed semantic segmentation based on a dual-camera system with different perspectives and overlapped views. Romera et al (2019), focused on the domain gap due to the moment of the day in which a given a scene has been captured.…”

Adverbial Time Indicators of Correlation of the Event with the Actual Moment: Structure, Semantics, Functions

“…If the unit of the language displays such variation, the communicator's awareness of the temporal properties of the event, when it is indispensable to establish the relationship between the execution time of the action and a certain orientation point, the appeal to the concept of «temporality» seems fair. A thorough study of the scientific experience of European (including Ukrainian) researchers (Barchuk, 2017;Belousenko, 2012;Bondar, 2018;Osipchuk, 2020;Saratsli et al, 2020;Stringer, 2020;Suzuki & Wheatcroft, 2020;Xiang et al, 2020), Those interested in the problem of an adequate interpretation of this term allows us to state that temporality is a «linguistic aspect of the category of time», the semantic category of narrow specialization, as well as the Monocentric functional-semantic field. The presence in the semantic principle of this "continuous spherical formation" of actualization specifics is evidenced by the formation by the speaker during the speech act of a kind of meaningful complex that contains "orientation to the initial position of actualization as a reference point".…”