An Intelligent In-cabin Monitoring System in Fully Autonomous Vehicles

Mishra, Ashutosh; Kim, Jinhyuk; Kim, Dohyun; Cha, Jaekwang; Kim, Shiho

doi:10.1109/isocc50952.2020.9333062

Cited by 8 publications

(8 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To enhance the safety and security inside the vehicle's cabin, multiple works have been reported [5][6][7][8][9][10][11][12][13][14][15][16][17]. Recently various companies have also reported their in-cabin monitoring systems for improving the safety and security of the occupants inside the vehicle [18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33].…”

Section: Related Workmentioning

confidence: 99%

“…Facial anonymization serves to effectively overcome this issue, as the anonymous face protects personal information during in-cabin monitoring. In our previous works, we have proposed an efficient algorithm for face anonymization of the occupants [5,6,52]. The generative adversarial network (GAN) is utilized for facial swapping and reenactment.…”

Section: Face Anonymization: Privacy-preserved Monitoring In Publicmentioning

confidence: 99%

“…Furthermore, the vehicle itself requires protection from any malicious behavior by the occupants. Therefore, multi-pronged in-cabin monitoring of level 4 and beyond AVs in real-time is crucial [5,6]. Plenty of past works have reported in-cabin monitoring for different purposes However, in-cabin monitoring in level 4 and beyond AVs should intend for the safety and security of the occupants and the vehicle [5,6].…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, multi-pronged in-cabin monitoring of level 4 and beyond AVs in real-time is crucial [5,6]. Plenty of past works have reported in-cabin monitoring for different purposes However, in-cabin monitoring in level 4 and beyond AVs should intend for the safety and security of the occupants and the vehicle [5,6]. The safety of each occupant implies their physical protection.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

In-Cabin Monitoring System for Autonomous Vehicles

Mishra

Lee

Kim

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

In this paper, we have demonstrated a robust in-cabin monitoring system (IMS) for safety, security, surveillance, and monitoring, including privacy concerns for personal and shared autonomous vehicles (AVs). It consists of a set of monitoring cameras and an onboard device (OBD) equipped with artificial intelligence (AI). Hereafter, this combination of a camera and an OBD is referred to as the AI camera. We have investigated the issues for mobility services in higher levels of autonomous driving, what needs to be monitored, how to monitor, etc. Our proposed IMS is an on-device AI system that indigenously has improved the privacy of the users. Furthermore, we have enlisted the essential actions to be considered in an IMS and developed an appropriate database (DB). Our DB consists of multifaced scenarios important for monitoring the in-cabin of the higher-level AVs. Moreover, we have compared popular AI models applied for object and occupant recognition. In addition, our DB is available on request to support the research on the development of seamless monitoring of the in-cabin higher levels of autonomous driving for the assurance of safety and security.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Face Anonymization: Privacy-preserved Monitoring In Publicmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

In-Cabin Monitoring System for Autonomous Vehicles

Mishra

Lee

Kim

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, Level 5 vehicles do not have an active human driver [ 5 , 6 ]. This requires a robust solution for directing the AV safely and seamlessly, under such challenging circumstances.…”

Section: Introductionmentioning

confidence: 99%

Authorized Traffic Controller Hand Gesture Recognition for Situation-Aware Autonomous Driving

Mishra

Kim

Cha

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

An authorized traffic controller (ATC) has the highest priority for direct road traffic. In some irregular situations, the ATC supersedes other traffic control. Human drivers indigenously understand such situations and tend to follow the ATC; however, an autonomous vehicle (AV) can become confused in such circumstances. Therefore, autonomous driving (AD) crucially requires a human-level understanding of situation-aware traffic gesture recognition. In AVs, vision-based recognition is particularly desirable because of its suitability; however, such recognition systems have various bottlenecks, such as failing to recognize other humans on the road, identifying a variety of ATCs, and gloves in the hands of ATCs. We propose a situation-aware traffic control hand-gesture recognition system, which includes ATC detection and gesture recognition. Three-dimensional (3D) hand model-based gesture recognition is used to mitigate the problem associated with gloves. Our database contains separate training and test videos of approximately 60 min length, captured at a frame rate of 24 frames per second. It has 35,291 different frames that belong to traffic control hand gestures. Our approach correctly recognized traffic control hand gestures; therefore, the proposed system can be considered as an extension of the operational domain of the AV.

show abstract