Vehicle localisation and deep model for automatic calibration of monocular camera in expressway scenes

Zhang, Wentao; Song, Huansheng; Liu, Lichen; Congliang, Li,; Mu, Bochen; Gao, Qian

doi:10.1049/itr2.12152

Cited by 8 publications

(4 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our model can be trained on any video sequence, so it cannot assume the presence of the desired targets in the scene for camera calibration. In contrast, targetless methods usually make strong assumptions about the geometric structure of the scene (Assadzadeh et al., 2021; Zhang et al., 2023). Additionally, the traditional learning‐based calibration methods use supervisory signals to train regressors, such as a deep neural network (Bogdan et al., 2018), which requires additional data for calibration.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Monocular 3D object detection for construction scene analysis

Shen,

Jiao,

Zhang

et al. 2023

Computer aided Civil Eng

View full text Add to dashboard Cite

Three‐dimensional (3D) object detection, that is, localizing and classifying all critical objects in a 3D space, is essential for downstream construction scene analysis tasks. However, accurate instance segmentation, few 2D object segmentation and 3D object detection data sets, high‐quality feature representations for depth estimation, and limited 3D cues from a single red‐green‐blue (RGB) image pose significant challenges to 3D object detection and severely hinder its practical applications. In response to these challenges, an improved cascade‐based network with a transformer backbone and a boundary‐patch‐refinement method is proposed to build hierarchical features and refine object boundaries, resulting in better results in 2D object detection and instance segmentation. Furthermore, a novel self‐supervised monocular depth learning method is proposed to extract better feature representations for depth estimation from construction site video data with unknown camera parameters. Additionally, a pseudo‐LiDAR point cloud method and a 3D object detection method with a density‐based clustering algorithm are proposed to detect 3D objects in a construction scene without help from 3D labels, which will serve as a good foundation for other downstream 3D tasks. Finally, the proposed model is evaluated for object instance segmentation and depth estimation on the moving objects in construction sites (MOCS) and construction scene data sets. It brings a 9.16% gain in terms of mean average precision (mAP) for object detection and a 4.92% gain in mask mAP for object instance segmentation. The average order accuracy and relative mean error for depth estimation are improved by 0.94% and 60.56%, respectively. This study aims to overcome the challenges and limitations of 3D object detection and facilitate practical applications in construction scene analysis.

show abstract

Section: Methodsmentioning

confidence: 99%

“…2-after squeeze era calibration. In contrast, targetless methods usually make strong assumptions about the geometric structure of the scene (Assadzadeh et al, 2021;Zhang et al, 2023).…”

Section: Layer Description Kernel Sizementioning

confidence: 99%

Monocular 3D object detection for construction scene analysis

Shen,

Jiao,

Zhang

et al. 2023

Computer aided Civil Eng

View full text Add to dashboard Cite

show abstract

“…However, it only identifies semantic features for the entire scene and does not utilize extensive geometric information, resulting in lower accuracy. Zhang et al [15] introduced a deep calibration network that estimates vanishing points and camera extrinsic parameters from RGB images in an end-to-end manner. Then, they calculate the camera focal length using the vanishing point and camera rotation angle based on perspective projection geometry.…”

Section: Deep Learning-based Camera Calibration In Traffic Scenesmentioning

confidence: 99%

“…Deep learning-based roadside camera calibration methods [13][14][15] are not restricted to road scenes and do not require any manual input or prior information, making them increasingly popular in traffic scenarios. These methods are not limited by scene conditions and can provide relatively stable results.…”

Section: Introductionmentioning

confidence: 99%

Automatic Roadside Camera Calibration with Transformers

Li,

Zhao,

Chen

et al. 2023

Sensors

View full text Add to dashboard Cite

Previous camera self-calibration methods have exhibited certain notable shortcomings. On the one hand, they either exclusively emphasized scene cues or solely focused on vehicle-related cues, resulting in a lack of adaptability to diverse scenarios and a limited number of effective features. Furthermore, these methods either solely utilized geometric features within traffic scenes or exclusively extracted semantic information, failing to comprehensively consider both aspects. This limited the comprehensive feature extraction from scenes, ultimately leading to a decrease in calibration accuracy. Additionally, conventional vanishing point-based self-calibration methods often required the design of additional edge-background models and manual parameter tuning, thereby increasing operational complexity and the potential for errors. Given these observed limitations, and in order to address these challenges, we propose an innovative roadside camera self-calibration model based on the Transformer architecture. This model possesses a unique capability to simultaneously learn scene features and vehicle features within traffic scenarios while considering both geometric and semantic information. Through this approach, our model can overcome the constraints of prior methods, enhancing calibration accuracy and robustness while reducing operational complexity and the potential for errors. Our method outperforms existing approaches on both real-world dataset scenarios and publicly available datasets, demonstrating the effectiveness of our approach.

show abstract

Geo‐spatial traffic behaviour analysis and anomaly detection for ITS applications

Akdag,

D'Amicantonio,

Vijverberg

et al. 2024

IET Intelligent Trans Sys

View full text Add to dashboard Cite

Understanding the behaviour of traffic participants within the geo‐spatial context of road/intersection topology is a vital prerequisite for any smart ITS application. This article presents a video‐based traffic analysis and anomaly detection system covering the complete data processing pipeline, including sensor data acquisition, analysis, and digital twin reconstruction. The system solves the challenge of geo‐spatial mapping of captured visual data onto the road/intersection topology by semantic analysis of aerial data. Additionally, the automated camera calibration component enables instant camera pose estimation to map traffic agents onto the road/intersection surface accurately. A novel aspect is approaching the anomaly detection problem by AI analysis of both the spatio‐temporal visual clues and the geo‐spatial trajectories for all type of traffic participants, such as pedestrians, bicyclists, and vehicles. This enables recognition of anomalies related to either traffic‐rule violations, for example, jaywalking, improper turns, zig‐zag driving, unlawful stops, or behavioural anomalies: littering, accidents, falling, vandalism, violence, infrastructure collapse etc. The method achieves leading anomaly detection results on benchmark datasets World Cup 2014, UCF‐Crime, XD‐Violence, and ShanghaiTech. All the obtained results are streamed and rendered in real‐time by the developed TGX digital twin visualizer. The complete system has been deployed and validated on the roads of Helmond town in The Netherlands.

show abstract

Vehicle localisation and deep model for automatic calibration of monocular camera in expressway scenes

Cited by 8 publications

References 47 publications

Monocular 3D object detection for construction scene analysis

Monocular 3D object detection for construction scene analysis

Automatic Roadside Camera Calibration with Transformers

Geo‐spatial traffic behaviour analysis and anomaly detection for ITS applications

Contact Info

Product

Resources

About